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The Meaning of Title VII 


‘ee JULY ISSUE OF News and Reports* describing more than 80 
Title VII grants comes as a reminder that almost two years of research 
on the new educational media under the National Defense Education 
Act is now behind us. What has Title VII accomplished? How is it 
influencing education? Where is it taking us? 

As far as tangible achievements are concerned, a large number of 
individual grants have been made for research, and many of the studies 
are at an advanced stage. We also have the contract program under 
Part B of Title VII which already has to its credit several important 
regional research conferences and a wide range of dissemination of 
research projects. 

The effects of Title VII on education is a little more difficult to assess. 
Obviously we cannot separate the influence of this program from that 
exerted by Ford Foundation activities or by other on-going research 
programs. It is however apparent that Title VII has stimulated re- 
searchers to concentrate their efforts on discovering relationships of 
educational media to instruction, thereby alerting the entire educational 
field to the implications of technological developments. This was a 
trend started by the Ford Foundation several years ago with its full- 
scale attack on the educational television problem. Title VII appears 
to be broadening the influence to include a wider array of media 
applications. 

Where is Title VII taking us? Here the answers are even less clear. 
At least in its grant program, Title VII is limited by its dependence 


* Published by the New Educational Media Branch, Office of Education, U.S. Department 
of Health, Education, and Welfare. 
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upon a system of voluntary project submission; the Office of Education 
cannot itself initiate the experimental study of necessary research. So 
it is not surprising to find almost half of the approved grants comparing 
the instructional effectiveness of the various media with conventional 
approaches to instruction. Nor is it surprising to note that about one- 
third of the grants have been made for the study of televised teaching. 
At the same time, an increase in the study of self-instructional ap- 
proaches to learning and the production variables that lead to the 
design of these tools are a promising recent trend. 

There remain as yet two significant gaps in the research coverage of 
the grant program, however. First, there is need for greater emphasis 
on the support of projects that investigate the application of total learn- 
ing systems utilizing a variety of media combined in accordance with 
their unique instructional characteristics and for specific learning ob- 
jectives. Second, some study is needed of factors relating to the adminis- 
tration and organization of education systems designed for the optimum 
use of all instructional media. Also worth considering is the fact that 
the possibilities of what may well be the most important instructional 
tool of all—the electronic computer—have as yet been almost com- 
pletely ignored. 

So the start that Title VII has made is encouraging. Its staff has 
been selflessly dedicated to the administration of the program. Hundreds 
of competent researchers are contributing to its success. The editors of 
AVCR wish to commend the Office of Education and the Advisory 
Committee on New Educational Media for their efforts in getting this 
difficult program in high gear. 

William H. Allen, Editor 
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Audiovisumatic Teaching: 
A New Dimension in Education and Research 


KARL U. SMITH 


This report follows Dr. Smith’s article in the Winter 1960 issue of 
AVCR. The research on audiovisumatic teaching reported here is an 
extension of the scientific principles of artistic illustration of teaching 
materials as treated in that article and as originally set down in THE 
BEHAVIOR OF MAN (Henry Holt & Co., 1958) by the author and William 
M. Smith. This research on audiovisumatic teaching has been made 
possible by support from the National Science Foundation for a project 
on Perception and Human Motion and by funds provided by the 
Graduate School Research Committee of the University of Wisconsin.’ 
A special report on the application of audiovisumatic teaching methods 
is available on request from the author. Dr. Smith can be reached at 
Indiana University where he is on leave from the University of Wis- 
consin as Ford Foundation professor of business administration. 


y on PAPER describes the development and preliminary evaluation of 
a new method and approach to classroom teaching and educational re- 
search, to which we have applied the term, “audiovisumatic teaching.” 
Audiovisumatic teaching is a new concept and a new dimension at 
all levels of teaching—in individual tutoring, quiz-sections, discussion 
groups, and formal lectures. As a method of automation of teaching 
functions, it can be utilized by the individual teacher or in a teaching 
center. It is a new way to apply visual art and illustration to all aspects 
of oral teaching. It also provides the essential instrumentation for illus- 
trated lecture review and machine control of programming and rein- 
forcement in self-study. 
+ Numerical data analyses of some aspects of this research was aided by the Numerical 


Analyses Laboratory, The University of Wisconsin. 
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The term “audiovisumatic teaching” refers to the interrelated re- 
cording and playback of illustrated lectures and student control of 
programmed tutoring material. It involves the presentation of mag- 
netic tape recordings and visual projection materials by means of 
flexible electronic controls. As adapted to self-study and review pur- 
poses, presentation of questions can be controlled in relation to answers 
given by the student. Audiovisumatic teaching devices maintain the 
“live” character of the oral lecture and automate presentation of 
supporting visual material. Far simpler and less costly to use than 
motion pictures or television in teaching, audiovisumatic methods 
probably achieve a more effective level and breadth of instruction than 
do these other specialized techniques. Yet audiovisumatic presenta- 
tions, as well as the question-and-answer materials used by the student, 
can be prepared, recorded and presented by the average classroom 
teacher with no specialized technical training. 

Audiovisumatic teaching makes possible many new techniques and 
practices in classroom procedure and educational research. The 
present study is concerned with an evaluation of some of these new 
methods and student attitudes toward them. 


AUDIOVISUMATIC TEACHING DEVICES 


The audiovisumatic teaching machine is a magnetic tape recorder 
with built-in electronic controls which can operate a conventional 
slide or filmstrip projector. In addition, the device contains an electronic 
control unit which is used to stop the tape recorder automatically 
after a question has been asked, and to start it again when the student 
correctly marks the answer to the previous question on an answer sheet. 

Figure I illustrates one of our first audiovisumatic teaching machines 
which combines both lecturing and teaching functions. It can give 
an illustrated lecture, with slides automatically presented at the proper 
time, and can also ask questions which must be answered correctly by 
the student before he can proceed. The machine can also direct the 
student to write an answer to a short essay question, the correct answer 
to which is summarized by the machine when the student finishes 
writing and punches his answer sheet for a second question. Generally, 
the machine can be made to automate any aspect of the teaching 
function except spontaneous give-and-take between lecturer and class. 
Even in this area, the machine can be used to record such spontaneous 
give-and-take, and play it back to the individual student or class. 
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FIGURE |. An audiovisumatic teaching machine. Composed of a magnetic tape re- 
corder, an electrically controlled slide projector, and an audiovisumatic control 
unit, the machine automates the lecturing function and the presentation of visual 
slide material. It also automatically feeds back information to the student for 
incorrect and correct responses to questions put to him by the machine. 


Figure II diagrams the circuit relations of the components of the 
audiovisumatic teaching machine. Two of these are conventional com- 
ponents, a tape recorder and a standard 2 x 2 slide or filmstrip pro- 
jector. The audiovisumatic control unit is a small amplifier-relay which 
is sensitive to very small signal voltages and will control fairly heavy 
load currents. This basic control relay has been perfected over a period 
of some years in our research on electronic methods of human motion 
analysis. As used in the audiovisumatic teaching machine, the relay 
can be activated by contacts placed on the magnetic tape to control the 
operations of the slide projector or the motor of the tape recorder. 
One circuit in the device is connected to the motor of the tape recorder, 
and to a special electrode stylus used by the student. Contacts on one 
side of the magnetic tape activate this signal circuit and stop the tape 
recorder. When the student marks an answer sheet correctly, the tape 
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FiGuRE II. The circuit relations of the component parts (magnetic tape recorder, 
slide projector, audiovisumatic sensing and control unit, and answering sheet). 
The machine automates the oral lecture by conventional playback technique. The 
slide projector is activated by means of contacts on the magnetic tape which pro- 
vide a signal to the audiovisumatic control unit: The feedback of information 
about correctness of answers is automated by using the contact between the stu- 
dent’s stylus and marked areas of the answer key to switch the signal circuit of the 
control unit and to start the tape recorder. 
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recorder starts up again. Thus the machine can be made to stop auto- 
matically after asking a question, and proceed with the next question 
after the student has punched the correct response on the answer sheet 
with his stylus. 


One of the big advantages of the audiovisumatic teaching machine 
is that the teacher can make up his own answer sheets and keys, as 
shown in Figure II. The alternative multiple-choice answers to the 
questions asked by the machine are printed on a sheet of paper by 
either ditto or mimeograph process, and answer spaces are drawn beside 
these statements. To make a key, the teacher merely blackens the 
spaces on one sheet corresponding to the correct answers. This sheet 
is placed below the answer sheet. When the student punches the answer 
sheet correctly, contact of his stylus with the graphite of the marked 
key (below the answer sheet) completes the circuit of the audiovisumatic 
control and starts the tape recorder which then asks another question 
and stops automatically. 


The same principles used in the audiovisumatic machines are em- 
ployed in the construction of visumatic and audiomatic teaching ma- 
chines. For the visumatic unit, lecture material and questions are put 
on film or slides. When a question is asked, the student must respond 
correctly in order to get another slide or question. 


In addition to the types of controls already described, the audiovisu- 
matic control unit can be connected to timers, counters, mechanical 
registers, or markers in order to obtain quantitative data on time, errors, 
and correct responses of students in relation to various methods and 
procedures of teaching. Thus audiovisumatic techniques can be used 
in educational research to evaluate variable elements of the teaching 
situation. Type of voice, rate of talking, difficulty of material, inte- 
gration of visual-aural material, programming of special courses, pre- 
testing of new courses—all of these can be tested quantitatively by 
audiovisumatic methods. 


PRELIMINARY EVALUATION 


The audiovisumatic and visumatic teaching machines have been 
given a preliminary evaluation in a series of lectures, lecture demon- 
Strations, and laboratory studies of lecture review and automated 
tutoring. In a class of approximately 90 students in beginning psy- 
chology a highly technical audiovisumatic lecture on hearing was 
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presented. In a regular class session two weeks later, some of the 
material from this lecture was used to demonstrate the lecture review 
and automated tutoring functions of the audiovisumatic teaching ma- 
chine. One month later, a two-part attitude survey was presented to 
the students in the course. A first part asked the students to list in 
order the three positive and the three negative features of audiovisu- 
matic teaching they considered most significant. In the second part of 
the survey, the students’ attitudes toward the machine were compared 
with their attitudes toward other teaching aids. 

A second evaluation of audiovisumatic teaching was conducted in a 
class of 45 students (juniors and seniors) in industrial psychology. A 
45-minute illustrated lecture on unemployment was prepared in which 
some 20 slides were used. Only four of these slides were actual draw- 
ings; the rest were handwritten outlines of subject matter of the lecture, 
which were correlated with the oral presentation by the machine. In 
the absence of the regular teacher, this audiovisumatic lecture was 
presented without comment by an assistant. Three weeks later, a dem- 
onstration of audiovisumatic teaching was given to the class in which 
the lecture review and automated tutoring functions of the machine 
were shown and possible uses in marketing, advertising, selling and 
industrial training described. Several weeks later, these students were 
presented with the two same general survey procedures used with the 
elementary students, except for some differences in the preference items 
in the second part of the survey. 

Four exploratory laboratory studies have been run on audiovisu- 
matic lecture review and automated tutoring. In the first, reaction 
time and number of correct responses to questions about lecture ma- 
terial following the initial presentation by a machine were compared 
with reaction time and correct answers after a single review. In a 
second study, two groups of subjects were compared in regard to two 
conditions of learning the material of a 20-minute industrial indoc- 
trination lecture. One group heard the audiovisumatic presentation of 
this lecture and the other read the same material from a printed manual. 
In a third study visual presentation of artistically illustrated material 
in psychology was compared with an entirely verbal presentation, pre- 
sented visually on slides by means of a visumatic machine. The level 
of information gained from the two methods of presentation was meas- 
ured in terms of the number of correct responses to the questions. In 
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the fourth study a similar comparison was made between oral and 
visual presentation of paired nonsense syllables. Reaction time and 
number of correct responses were measured by means of the audiomatic 
and visumatic controls. 


Critical Positive Attitudes 


In the first part of the attitude survey students were told to list the 
three critical features of audiovisumatic teaching which for them best 
defined the advantages and the three which best defined the disadvan- 
tages. The number of times that a given critical factor was listed was 
determined. 

Table 1 shows those positive factors listed at least five times by 
students in elementary psychology. Out of a total of 204 responses of 
positive factors, 21 fell into categories with frequencies below five, and 
are not shown. The positive factors listed by advanced students at 
least four times are shown in Table 2. Out of a total of 120 factors 
listed, 25 fell into categories with frequencies below four and are not 
shown. The wording of these factors follows the word usage of the 
students as closely as possible. 

Tables 1 and 2 show that relating vision and hearing by the audio- 
visumatic teaching machine appears at the top of the list for beginning 
students and second in frequency for advanced students. Beginning 


TABLE 1—CRITICAL POSITIVE FACTORS LISTED FOR AUDIOVISUMATIC 
TEACHING BY BEGINNING STUDENTS 











Factors Frequency Percent 
Lecture and material related 39 19 
Can be replayed (lecture review) 23 11 
A good learning device assuring understanding 15 8 
A quiz giving your own knowledge at once 14 7 
Better organized material 12 6 
Sticks to or emphasizes subject 10 5 
Good visualization or visual supplement 9 4 
Optimum type of teaching device 8 4 
Substitute for reading 8 4 
Increases efficiency in time and work 8 4 
Improves learning with voice and vision 8 4 
Holds attention 7 3 
Lectures at your leisure—a private lecturer 6 2 
Combines sound and sight that a lecturer can not 6 2 
Better lectures 5 2 
Eliminates distracting mannerisms of the lecturer 5 2 


Total number of responses—204 
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TABLE 2—CRITICAL POSITIVE FACTORS LISTED FOR AUDIOVISUMATIC 
TEACHING BY ADVANCED STUDENTS 














Factors Frequency Percent 
Can be used to bring experts to class 33 27 
Correlates vision and lecture 20 17 
Used to perfect lectures 12 10 
Lecture more complete or better organized 7 6 
Reduces load on instructor 6 5 
Attracts attention better 5 4 
Increases variety in class 4 3 
Can be used when lecturer is absent 4 3 
Changes atmosphere of classroom 4 3 


Total number of responses—120 


students emphasized the values of automated tutoring and the lecture- 
review functions, whereas advanced students strongly emphasized its 
value in bringing the recorded lectures of great men and of experts 
into the classroom. 

Factors mentioned by the beginning students but not given in the 
table were: greater range of lecture material; interesting new method; 
an effective method for dry courses not offset by the lecturer’s person- 
ality, or for factual courses. Factors mentioned by advanced students 
but not given in the table were that audiovisumatic teaching constricts 
the time and distance for the classroom, that lectures can be repeated, 
that the method can be used for parts of the lecture, and that it is 
easy to take notes. 


Critical Negative Factors 


The negative factors listed by elementary and advanced students 
for audiovisumatic teaching are given in Tables 3 and 4. Listed most 
frequently by elementary psychology students are the rigidity of ma- 
chine teaching, its impersonal nature, the fact that the main lecture 
given was too fast (which it was), and that the machine cannot be 
reversed for quick review. Negative factors not shown in Table 4 were 
the machine’s failure to inspire constructive thinking, the fact that it 
might get fouled up, its expense, and its difficulty of use. 

The negative factors listed most frequently by advanced students 
are similar to those of the elementary students, namely, that the ma- 
chine gives one no chance for his own questions, that it is too imper- 
sonal, that it doesn’t hold attention as well as a live lecture, and that 
it might be misused. Other factors mentioned by these advanced stu- 
dents were that the machine must still be operated by someone, that 
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TABLE 3—CRITICAL NEGATIVE FACTORS LISTED FOR AUDIOVISUMATIC 
TEACHING BY _ELEMENTARY _PSYCHOLOGY STUDENTS 











F actors Fr requency Percent 
Can't raise specific questions of your own 31 16 
Too impersonal; doesn’t have human effect 31 16 
Lecture given was too fast * 24 12 
Will not permit rapid return to earlier point 15 7 
Could become boring; or, lacks interest 9 5 
Lecturer too technical * 9 5 
Not as convenient or as good as book 8 4 
Hard to remember things by machine 7 4 
Too mechanical (degrades student) 7 + 
Monotony of voice and rate on machine 6 3 
Can't go at your own rate 6 3 
Note-taking difficult 5 3 
Can't replace books 5 3 
Not as good as live lecture 5 3 


Total negative eepente—878 








“This was a highly technical lecture on » hearing that was intended to pressure 
students to the point of being frustrating. 


TABLE 4—CRITICAL NEGATIVE FACTORS LISTED FOR AUDIOVISUMATIC 
TEACHING BY _ADVANCED STUDENTS 














F actors F requency _ Percent 

No chance for questions of. your own 32 28 
Too standardized and impersonal 24 21 
Doesn't hold attention as well as lecturer 10 9 
Might be misused 7 

Too fast and “production-like” a 3 
Mechanical failure would spoil lecture a 3 
Possibility of boredom great 4 3 
No repetition to clear up points a 3 
Novelty of machine may wear 4 3 


Total riegative responses—1 15 














the sound was not clear, that the machine would lead to class cutting, 
that it would not be possible to use such teaching as a course. Neither 
group of students listed as many negative factors as positive ones. 


Preferences of Students for Audiovisumatic Teaching Procedures 


In the elementary psychology class a three-item attitude scale was 
used which permitted comparison between the use of audiovisumatic 
teaching, an artistically illustrated book (1), and an artistically illus- 
trated workbook (2). The results on these items are given in Table 5 
in terms of the frequency and percentage (given in parentheses) of 
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students who strongly agreed, agreed, or were in doubt about the altern- 
ative preferences in each item. 

The book used by this class is a highly illustrated textbook which 
has been judged by over 85 percent of two large classes in elementary 
psychology to be well organized and interesting. Table 5 shows that 
33 percent of these elementary students either agreed or strongly 
agreed that they preferred the use of an audiovisumatic machine to a 
book of this sort for study. The workbook used in the course is an 
artistically programmed workbook, generally liked by students, as 
judged both by attitude surveys and by comments. Fifty-seven percent 
of the students in this study either agreed strongly or agreed that they 
would prefer audiovisumatic teaching methods to the use of a workbook. 

On the general item of preference for use of audiovisumatic teaching 
in a course, 78 percent of the students agreed or strongly agreed that 
they would like the machine sometimes in class. 


TABLE 5— PERCENTAGE OF STUDENTS (IN PARENTHESES) SHOVING 
DIFFERENT INTENSITIES OF ATTITUDE TOWARD 
EDUCATIONAL TECHNIQUES (N=78) 


Item Strongly Agree Agree Doubt 

Prefer book (48) 38 (13) 10 (3) 2 
Prefer machine (14) 11 (19) 15 (3) 2 
Prefer machine (17) 13 (40) 31 (0) O 
Prefer workbook (18) 14 (22) 18 (3) 2 
Would not care for machine at all (15) 11 (S) 4 (0) O 
: 


Would like machine sometimes (39) 29 (39) 29 (3) 


TABLE 6—ATTITUDES OF ADVANCED STUDENTS TOWARD AUDIOVISU- 
MATIC TEACHING (N=42) 


Item Strongly Agree Agree Doubt 
Would like machine sometimes in class (48) 20 (43) 18 (9) 4 
Would not be interested in having it in class (O) O (O) O (0) O 
Would like complete audiovisumatic course fap (29) 12 (38) 16 
Would never take audiovisumatic course fe igo (17) 7 7p 3 


Would like audiovisumatic center at University 


of Wisconsin (50) 21 (38) 16 fiz} 3 
Would not like audiovisumatic center at University (0) 0 (0) O (0) O 
May be made as good as live lectures fer. 3 (zi; 9 (29) 12 
Can’t be made as good as live lectures (20) 8 (26) 11 (2) 1 


May be made as good educationally as 

motion pictures (20) 8 (46) 19 (14) 6 
Can’t be made as good educationally as 

motion pictures 


(2) 1 (9) 4 (9) 4 
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The students in the industrial psychology class were given a five-item 
preference scale, as shown in Table 6. Ninety-one percent indicated that 
they would like the machine sometimes in class; none indicated that 
they would not. Thirty-one percent agreed or strongly agreed that they 
would like a complete audiovisumatic course; 24 percent said they 
would not take such a course. Eighty-eight percent noted they would 
like an audiovisumatic teaching center in the University; none noted 
they would not. Twenty-three percent thought that audiovisumatic 
teaching could be made as good as live lectures; 46 percent thought 
that it could not. Sixty-six percent agreed or strongly agreed that audio- 
visumatic teaching could be made as effective as motion pictures; 11 
percent thought it could not. 


Effect of Audiovisumatic Lecture Review 


Measurements were made of the effects of audiovisumatic lecture 
review on both errors and reaction time to objective questions presented 
by the machine. In order to obtain highly conventional and easily 
understood material for these lectures, we used interesting, well-known 
advertisements about cameras, liquor, ladies’ garments, travel, sports 
equipment, projectors, and so forth. Twenty of these samples were 
recorded on an audiomatic tape by the writer. Ten were one-minute in 
length, and ten were two-minute talks. The two types of talks were given 
alternatively, starting with the shorter. At the end of each talk, a 
single question was asked about some major fact presented in it, after 
which the machine stopped automatically until the correct answer was 
marked on a three-choice multiple-choice answer sheet. A precision 
time clock activated by the machine measured the reaction time to each 
question. An experimenter recorded the clock readings after each 
question was answered. Errors were determined by counting the number 
of incorrectly punched alternatives on the answer sheet. 

Eighteen college students were used as subjects in the experiment, 
working individually with the audiovisumatic machine. They were re- 
warded for participation by extra examination points toward their final 
grade in the course. Each subject heard the lectures and answered the 
questions in an initial experimental period, and two days later the same 
procedure was repeated in a review period. Thus the results show the 
effect of a single review of the lecture as well as the effect of repetition 
of questions upon errors and reaction times. 





96 AUDIOVISUAL COMMUNICATION REVIEW 


The total number of error choices marked for each question on the 
two-minute talks is summarized in Figure III. None of these questions 
were answered correctly by all 18 subjects in the first session. In the 
second session, all but one of the subjects answered questions 6 and 12 
correctly. The frequency of errors on some questions in the first learning 
was almost equal to the frequency of correct choices. Review sharply 
reduced the frequency of errors, usually by more than 50 percent. 
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FicureE III. Frequency of choices marked by 18 subjects in an initial learning 
period and in a review period two days later. The minimum choice frequency pos- 
sible for each question is 18—the number of correct responses among the 18 
subjects. The frequency of choice above 18 represents error frequency. 
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FiGurReE IV. Total reaction time of 18 subjects to each of 20 questions in the initial 
learning periods (black bars) and review periods (open bars). The data show that 
review has the effect of sharply reducing the response time of students in answering 
specific questions based on lectures. 


Results equivalent to these were found on error choices to questions 
based on the one-minute talks. 

Similar results were found for response time to each question. Figure 
IV summarizes the total reaction time of the 18 subjects to each of the 
10 questions based on the two-minute talks. Time of response on half 
the questions (Nos. 4, 6, 16, 18, 20) was reduced by 50 percent or more 
between the initial learning and review questions. The response-time 
data for questions about the one-minute talks in general duplicated these 
results found for the two-minute talks. 

Tests of significance of the differences in the error scores and time 
scores between the initial learning and review sessions show that both 
of these differences are significant at the 1 percent level. 
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Comparisons of Different Methods Presentation * 

In the first of three studies made in this area, audiovisumatic presenta- 
tion of job indoctrination material was compared with presentation of 
the same material in printed manual form. The 24 student subjects, 12 
who held residence-hall jobs and 12 who were applicants for such jobs, 
were divided into two experimental groups. One group was trained 
audiovisumatically and the other trained with printed material. Analysis 
of the data indicated that there was no significant difference between the 
mean errors made by each group in responding to questions covering 
the learning material. The main result of this research was to show that 
audiovisumatic trainers can be used as effectively for job indoctrination 
training as printed training material. 

In two additional studies in this series, visual and aural techniques of 
presenting learning material were investigated. Among a group of 20 
subjects, no differences in response time or in errors were found in 
learning associations between visually and aurally presented materials. 
The final study of this series showed that students to whom non- 
illustrated printed training material was presented visumatically an- 
swered questions equally well as those who were given a highly illus- 
trated presentation of this same material. 

The three studies just described have demonstrated that the audio- 
visumatic teaching machine is a flexible research instrument, useful in 
many kinds of educational research. The results of one study indicated 
that learning indoctrination material with the machine produced as 
accurate immediate memory as learning from a printed manual. In 
using the machine for comparing different types of visual and aural 
presentations, we have found no marked group differences in retention 
correlated with the various controlled methods of communication. 


THEORY AND DISCUSSION 


The development of audiovisumatic teaching devices described here 
has been guided by an overall theory of design of teaching procedures 
and methods, including textbooks and workbooks. In this so-called 
“perceptual-geometric theory” of symbol learning, the central notion is 
that learning in reading and listening is motivated and reinforced by 


*Richard Pinsker, Philip Jurgens, John Carlson, and Patrick McCann assisted in these 
stages of the research. 
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systematically promoting active use of words in relation to the 
perception of objects and space. 

The first main point of this space-organized theory of symbol learning 
is that learning in reading and listening is an active process, aided and 
regulated by nonverbal perceptual experience which provides a base of 
meaning for symbols and verbal abstractions. This perceptual organi- 
zation in the classroom can be aided by the nonverbal behavior of the 
teacher, by demonstrations, by graphs, by laboratory exercises, by 
object displays, and especially by artistically designed illustrative 
material. It can also be aided in the classroom, in lecture review, and 
in individual study, by audiovisumatic teaching devices. 


The second point of the theory is that visual art and illustration do 
not necessarily enhance immediate verbal memory, but that they 
motivate and broaden the active use of language and deepen insight by 
the student into the different dimensions of abstraction and thought. 
Well designed artistic illustration promotes understanding of the 
esthetic, moral, pragmatic, and emotional values in any learning 
situation, as well as the purely logical, scientific, or abstract concepts 
of language. Artistic illustration is an age-old, universal form of non- 
verbal communication that formed the bases of written language at its 
beginning, and which still is an essential accompaniment of the effective 
use of abstract symbols. We have applied this principle in designing the 
audiovisumatic teaching machine, which makes possible the economic 
use of artistic illustration in all phases of classroom teaching. 


A third principle of our perceptual-geometric theory of symbolism, 
which has been followed in the present development of audiovisumatic 
teaching devices, is that effective classroom illustration involves the use 
of correlated artistic design and symbolism. We mean that different 
forms of artistic drawing—representative, abstract, and impressionistic 
—can be used to enhance the motivation and level of creative thought 
of the student. Representative illustrations clarify specific object 
relations. Abstract drawings, along with the line graphs, deepen 
understanding of interaction of events, especially human and social 
interactions which cannot be quantified. Impressionistic designs 
promote understanding of object relations in terms of individualized 
emotional and perceptual values. It is our belief that the audiovisumatic 
teaching machine may be a more effective device than the motion picture 
or television in bringing correlated art into the classroom. 
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Recognizing the significance of reinforcing informational feedback 
to the student, in classroom learning or individual study, the present 
theory emphasizes that this feedback should involve organized per- 
ceptual responses on the part of the learner. Specifically, the perceptual- 
geometric theory of symbol learning assumes that visual illustration 
serves to act both as a motivating and as a reinforcing agent in reading 
and listening at three levels: (1) in providing a perceptually organized 
environment that creates familiarity with what is read and heard; (2) 
in directly reinforcing unfamiliar words and phrases by giving nonverbal 
behavioral cues of meaning for what is heard and read; and (3) in 
promoting creative thought about interaction of events through use of 
varied correlated forms of artistic expression—representative, abstract 
and impressionistic—to enhance the meaning of verbal symbols. 

The final point of the present theory is that most, if not all, the 
communicative phases of classroom teaching can be operationally 
specified in terms of (1) the functional levels of teaching (lecture, 
lecture review, and individual tutoring or instruction), (2) the com- 
municative behavior of written and oral speech, and (3) the nonverbal 
forms of emotional and artistic expression. Functional levels of teaching 
do not differ in terms of the words taught but in terms of the nonverbal 
activities used in lecturing, lecture review and in individual instruction. 
The audiovisumatic teaching machine is designed to automate all three 
of these functional levels of teaching and all phases of verbal and non- 
verbal expression in each of them, except the nonverbal gesture and 
emotional expression of the teacher. Thus, the audiovisumatic machine 
is designed not only to carry out the activities of teaching in an effective 
way, but to permit control of teaching functions and communications for 
research purposes. 

The theory outlined above was confirmed by some of the results of 
the present research. In our survey of self-formulated opinions about 
audiovisumatic teaching methods among elementary and advanced 
psychology students, the relating of vision and hearing was placed near 
the top of the list of positive characteristics of the teaching machines. 
This preference conforms to our general emphasis on the importance 
of organized perceptual responses in the learning situation. Further, 
the students’ negative opinions concerning the lack of individual partici- 
pation in audiovisumatic teaching and its impersonal character also 
conform to our general theoretical formulations. The audiovisumatic 
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machine was acceptable to the students in so far as it promoted active, 
organized perceptual responses, and unacceptable in so far as the 
students felt limited in their participation. 

The overall effectiveness of audiovisumatic teaching can be judged in 
terms of how well it conforms to desirable educational procedures as 
well as certain other practical considerations. In general we feel that 
the audiovisumatic machine is a flexible device which can be used at its 
best to provide highly varied, stimulating, and provocative classroom 
situations. Some of the different uses to which this machine can be 
adapted are the following: the use of recorded lectures and selected 
visual materials by experts in special fields and other well-known 
persons (the advanced students listed this as a highly desirable charac- 
teristic); interplay between the teacher and recordings of his own 
lectures; teacher-machine discussions; illustrated lecture review; various 
types of student-controlled programmed tutoring; use of recordings of 
related sounds and sights in nature; and playback of visually illustrated 
sounds of music and human communication. 

An important consideration in assessing the significance of audio- 
visumatic devices is how they compare with motion pictures and tele- 
vision in classroom use. From the teacher’s point of view, the audio- 
visumatic machine does a better job of teaching because it duplicates 
so well the normal teaching functions and has a flexibility in use not 
achieved by motion pictures or television. The audiovisumatic trainer 
promises to be a very economical device, both to build and to use. 
Made up of conventional components, a magnetic tape recorder and 
slide projector, plus an electronic control unit, it probably can be 
reduced in size from the present developmental units, and produced in 
quantity for about $300. The materials needed to prepare lectures, 
magnetic tapes, slides, and filmstrips are cheap and easily available. 
Teachers need no technical training either to prepare their own illus- 
trated lectures, to run the machines, or to prepare programmed material 
for individual tutoring. 


The audiovisumatic teaching machine is distinctive in providing for 
the use of standard answer sheets for controlled reinforcement in 
programmed tutoring. Answer-controlled reinforcement for both aural 
and visual questions is possible, as well as for aural questions directed 
toward visual displays. The great advantage here is that the teacher 
can make up his own answer sheets, quickly and easily, just as he can 








102 AUDIOVISUAL COMMUNICATION REVIEW 


prepare his own coordinated aural-visual lectures, reviews, and 
questions. This personal control by the individual teacher of his 
automated teaching materials cannot be duplicated by motion picture 
and television techniques. 

Audiovisumatic methods are applicable to many areas of teaching 
besides those investigated to date. Others include: (1) the development 
of integrated audiovisumatic lecture services for liberal studies; (2) the 
development of scientific and mathematical teaching services for the 
general public; (3) audiovisumatic aids to training in engineering, 
medicine, and other specialized fields; (4) use of audiovisumatic 
procedures for television presentation and control; (5) development of 
more effective teaching devices for the blind and the deaf; (6) use of 
audiovisumatic techniques in industrial training, advertising, marketing, 
and selling procedures; (7) development of community teaching centers 
utilizing audiovisumatic materials; (8) improvement and broadening of 
correspondence teaching; (9) providing audiovisumatic teaching serv- 
ices for underdeveloped nations; (10) setting up audiovisumatic 
teaching exchanges in order to broaden the use of knowledge of foreign 
scholars; and (11) expanding teaching services for hospital and military 
isolates. 

In addition to its use as a teaching machine, the audiovisumatic de- 
vice provides an interesting technique for future educational research. 
Various communication processes of teaching, verbal and nonverbal, 
aural and visual, can be automated and are subject to flexible control. 


SUMMARY AND CONCLUSIONS 


1. This paper describes the theory, development, application, and 
evaluation of a new procedure in education which we have termed 
audiovisumatic teaching. 

2. Audiovisumatic teaching machines automate all levels (lecturing: 
lecture review; and answer-controlled, programmed tutoring) and all 
major visual and aural patterns of communication in teaching. 

3. The audiovisumatic devices originally developed in this research 
are made up of a magnetic tape recorder, a slide projector, and an 
electronic control unit, which can be operated by the teacher or student. 

4. A distinctive feature of these machines is that they make use of 
standard answer sheets in machine-control or reinforcement feedback 
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to the student in programmed tutoring and can be used by the average 
teacher without technical assistance. 

5. A series of studies indicated that students not only have strong 
motivation to use audiovisumatic methods of teaching at the university 
level, but that they prefer these methods to some other methods of 
instruction. Two studies on self-formulated student attitudes confirm 
our views that the values of audiovisumatic methods depend on their 
effectiveness in (a) relating visual art and verbal material, (b) provid- 
ing immediate informational feedback to the student in lecture review 
and programmed tutoring, and (c) increasing variety of perceptual 
communication and review of classroom material. 

6. The results of one laboratory study indicated that audiovisumatic 
lecture review greatly increases accuracy and speed of responses to 
lecture questions. 

7. Three studies indicated that audiovisumatic techniques promote 
a high degree of control of avenues of communication in instructional 
research and that such methods are as good as using printed material in 
job indoctrination training. 

8. Several future applications of audiovisumatic methods in univer- 
sity teaching, industrial training, advertising, correspondence teaching, 
and training of human isolates are indicated. 
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Effectiveness of Inserted Questions 


in Instructional Films 


BERNARD R. KANTOR * 


A HUNDRED experimental 
studies have been made in the past 40 
years on the influences and effectiveness 
of instructional motion pictures. To- 
day, the teacher is faced with the prob- 
lem of teaching an_ ever-increasing 
amount of knowledge in a short time. 
With this dilemma facing the classroom 
teacher (1), research data dealing with 
the motion picture have an increased 
practical and theoretical importance. 

Various studies have indicated an in- 
creased efficiency for the instructional 
film when it is properly used by the 
classroom teacher. Stopping the film at 
the end of sequences and discussing the 
content is considered good teaching 
technique in using films. Noting im- 
portant points in the film and listing 
questions to be answered by the film 
are others (5). 

All of these techniques work to maxi- 
mum advantage, however, when the 
teacher has previewed the film in ques- 


* Bernard R. Kantor is assistant profes- 
sor of cinema at the University of Southern 
California. 


tion. This procedure is rarely possible 
with the present-day demands on the 
teacher and the requirements of film 
distribution. There have been some sug- 
gestions from time to time that ques- 
tions properly inserted within the body 
of the film might lead to greater teach- 
ing efficiency. Such questions might be 
motivational or aid in better mental 
“set.” This study is an attempt to seek 
clarification in this area. If this and 
further logical steps prove fruitful, the 
teacher might be relieved of the respon- 
sibility of always previewing a film be- 
fore showing it to a class. 


Hypothesis 


This study attempted to verify experi- 
mentally the hypothesis that inserted 
questions can significantly raise the 
teaching effectiveness of a factual in- 
structional film when such questions are 
inserted both visually and aurally. An- 
swers to the following specific questions 
were sought: 

1. Is there a gain in teaching effective- 
ness of an instructional film where ques- 
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tions are inserted into the film, visually 
and aurally, at natural divisions of the 
content? 

2. Where should the questions be 
placed in such a film? (a) What is the 
effect on learning when the question is 
placed before the material about which 
it is concerned? (b) When it is placed 
after the material? 

3. Will questions inserted into this 
film aid in greater retention of the ma- 
terial as measured by delayed recall 
tests than could be obtained without 
such questions? 

4. Are there any differences in teach- 
ing effectiveness of a film with questions 
inserted as compared to no questions, 
when results are grouped by IQ, sex, 
age, and socio-economic status? 


Design of Study 


The first task was the selection of a 
suitable film from which to make the 
experimental versions. A factual film 
which was highly regarded and consid- 
ered a “good” film by virtue of wide 
acceptance and use was The Sunfish, 
which was made available for the ex- 
periment by the Encyclopaedia Britan- 
nica Films, Inc. The population for the 
study was 617 second semester seventh 
grade pupils in three junior high schools 
in widely differing socio-economic areas 
of Los Angeles. Data available on each 
were (1) scores on the California Test 
of Mental Maturity taken within six 
months of the start of the study, and 
(2) occupation of the pupils’ parents. 

Three versions of the film, The Sun- 
fish, were prepared. These included a 
control version and two experimental 
versions. In one experimental version 
the questions were inserted before the 
content concerned; in the other, identi- 
cal questions were inserted after the 
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content; 
questions. 


the control version had no 


The eight questions used in this ten- 
minute film were photographed in such 
a manner that the quality was in keep- 
ing with the main body of the film. The 
narration was rerecorded along with the 
questions so that the same voice would 
cover both. The control version of the 
film was rerecorded with this same nar- 
ration but without questions. Thus the 
attention-getting device of a sudden 
change in voice was eliminated. 

On the basis of a pilot study, two 
test instruments were created, each hav- 
ing approximately the same reliability 
and measuring essentially similar con- 
tent. Questions on these instruments 
were arranged in increasing order of 
difficulty and discrimination. One of 
these instruments became the pre-test, 
one the post-test. A combination of the 
two served as the retention test. 

In each of the three schools, assign- 
ment of the films were made to permit 
equal numbers of classes, IQ groupings, 
sexes, and so on, to see each version. 
In each school, each film version was 
viewed by approximately 70 pupils (two 
classes of 35 each). 

All of the pupils took the pre-test; 
one week later they saw one version of 
the film in their own classroom and took 
a post-test; five weeks after the post- 
test a retention test was administered. 
Preparation of each class was accom- 
plished by reading a script containing all 
necessary instructions in order to in- 
sure the same «preparation for each 
group. Data collected and recorded 
were age, sex, language index, socio- 
economic status based on a modified 
W. L. Warner occupation scale (4, 3), 
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test scores, film version seen, and school 
attended. Data were analyzed by analy- 
sis of variance and covariance. 


Findings 


Analysis of the population data indi- 
cated that the desired spread of popula- 
tion was more than adequate. There 
were sufficient numbers of pupils of 
high, low, and average intelligence and 
socio-economic status for the require- 
ments of this study. There was a rela- 
tively equal number of boys and girls. 
Assignment of the film versions allowed 
for sufficiently matched groups to make 
competent analysis of the test results. 
The analysis follows: 

1. On the pre-test, boys did signifi- 
cantly better than girls, which may be 
ascribed to a greater initial interest in 
the subject matter. Other than on this 
pre-test, there were no reliable differ- 
ences between boys and girls for any 
of the film versions. (See Table 1.) 

2. Pupils with high language index 
did significantly better on each of the 


film versions than those with low lan- 
guage index. 

3. In each film version, pupils in 
high socio-economic status did signifi- 
cantly better than pupils in average 
socio-economic status, who, in turn, did 
significantly better than those in low 
socio-economic status. 

4. As might be expected, there was 
virtually a “classical” arrangement in 
age, language index, and socio-economic 
status. Pupils in low socio-economic 
status had the lowest language indexes, 
were the oldest in grade, and were low- 
est in test results for all film versions. 
Pupils in high socio-economic status 
had the highest language indexes, were 
the youngest in age, and had the best 
scores on pre-test, post-test, and reten- 
tion test for all versions of the film. 
Average-status pupils were average in 
language index, in age, and in test re- 
sults for all film versions. All differ- 
ences in such cases were very signifi- 
cant. As might be expected, although 
there were individual exceptions to the 


TABLE 1—MEANS AND STANDARD DEVIATIONS OF TEST RESULTS FOR 
PRE-TEST, POST-TEST, AND RETENTION TEST, FOR 
TOTAL GROUP TAKING EACH TEST 


Variable Group 


Boys 
Pre-test Girls 
Total 
(t=3.36) 
Boys 
Post-test Girls 
Total 
(t=1.89) 
Boys 
Retention * Girls 
test Total 
(t= .70) 


Note: The pre-test and post-test contained 33 items each. The retention test con- 


tained 66 items. 


Standard 
N Mean Deviation 
278 15.36 cm 
305 14.36 3.47 
583 14.84 3.62 
(p< .0O1) 
276 25.90 4.15 
300 25.29 Pe 
576 25.58 3.87 
(p< .10) 
254 45.46 7.59 
273 45.01 7.13 
9 a | 45.22 aD 
(p< .50) 
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TABLE 2—MEANS AND STANDARD DEVIATIONS FOR THE THREE FILM 
VERSIONS FOR FIVE VARIABLES FOR THE TOTAL POPULATION 


























Group Standard 

Variable (Film Version) N Mean Deviation 
Control 201 111.70 17.46 
Language Before Content 179 112.11 19.26 
Index After Content 196 112.43 15.63 
Total 576 112.08 17.43 

i (F= .09) (p=NS)* (2.573 df)” 

Control 201 150.82 6.73 
Age in Before Content 179 150.60 6.47 
Months After Content 196 150.22 5.74 
Total 576 150.55 6.32 

(F= .45) (p=NS) (2.573 df) 
Control 190 14.66 3.65 
Before Content 170 14.89 3.95 
Pre-test After Content 182 15.20 3.21 
Total 542 14.92 3.60 

(F=1.05) (p=NS) (2.539 df) 
Control 201 25.67 4.19 
Before Content 179 25.49 3.99 
Post-test After Content 196 25.58 3.40 
Total 576 25.59 3.87 

(F= 50) (p=NS) (2.573 df) 
Control 188 45.45 7.13 
Retention Before Content 165 44.52 7.92 
test After Content 174 45.65 7.02 
Total 527 45.22 7.35 


(p=NS) (2.524 df 


Note: The pre-test and post-test contained 33 items each. The retention test con- 


tained 66 items. 


‘ ®™NS=Not Significant. "(2.573 df) =2 and 573 degrees of freedom, etc. 


foregoing, the results were true for the 
total groups. 

5. In view of the above alignment, 
the differences between the three 
schools naturally followed the same pat- 
tern: (a) The school in the generally 
high socio-economic area had the 
youngest pupils, with highest language 
indexes, and the highest test scores on 
all film versions. (b) The school in the 
low socio-economic area was low in all 
instances and had pupils of the oldest 
age group. (c) The average school, in 
the average socio-economic area, was 
average in all instances. 


6. The total test results, as noted in 


Table 1, showed that the films did teach. 
The gains, as measured by post-test over 
pre-test scores, were highly significant 
in all cases, without a single exception. 
The same relative gain held for the re- 
tention test five weeks later, with a 
slight absolute loss, probably through 
forgetting. 

7. Analysis of the test results as 
shown in Table 2 produced no signifi- 
cant differences in pre-test, post-test, or 
retention test by film version. All ver- 
sions taught equally well for the total 
group. 

8. Even when the data were analyzed 
for the complete cases only (498 pupils 
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who had taken all three tests, had seen 
one film version, and for whom com- 
plete cumulative record data were avail- 
able), there were still no significant dif- 
ferences in film versions. All film ver- 
sions taught equally well. 

9. When film version data were ana- 
lyzed by sex, school, socio-economic 
status, language indexes, or age, a scat- 
tering of differences were noted favor- 
ing one film version or another. How- 
ever analyzing these sporadic differ- 
ences by covariance, and incorporating 
initial differences in language index 
usually reduced any differences to a not 
significant state. 


Conclusions 


The original and basic hypothesis for 
this study —that inserting questions, 
both visually and aurally, would aid in 
greater learning from a film than would 
be possible from a film without such 
questions—is rejected. Pupils tested in 
this study, regardless of age, sex, socio- 
economic status, or language index, 
learned equally well from all three film 
versions. Further, the amount of learn- 
ing retained, as measured by a reten- 
tion test, was the same for each of the 
three film versions. 


Each of the original questions relat- 
ing to the main hypothesis can be an- 
swered in the negative. It seems to make 
no significant difference if questions are 
inserted before or after the content, or 
if no questions are used at all. In this 
case, the results differ from the sugges- 
tions by several other researchers (2). 
Their suggestion that insertion of ques- 
tions should prove effective and there- 
fore might well be adopted by film pro- 
ducers can be questioned, although there 
is another factor that enters in. 


In a study such as this, consideration 
must be given to the size and nature of 
the sample when drawing conclusions. 
This study considered only seventh 
graders and hence it does not neces- 
sarily follow that inserted questions 
might not be effective with younger or 
older children. Repetition under differ- 
ent and better control, with different 
grade levels, might result in different 
findings. 

Inserted questions in a foreign lan- 
guage film, where the question is in the 
language being studied and the answer 
is presented in printed or visual form 
or both, might be an interesting and 
worthwhile area for investigation. The 
attention-getting device of the question 
being spoken by a different voice from 
that of the film narrator is another pos- 
sible area of study. Obviously, much 
more study is needed in the instructional 
film area, in the entire area of learning 
and, particularly, the dimensions of 
viewing. 
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Can Subliminal Stimuli Teach? 


EUGENE K. OXHANDLER * 


I, THERE a more economical way to 
communicate facts, ideas or concepts 
than some of the more conventional 
methods now in use? Is there some 
method for making present communica- 
tive media more effective? Certainly an 
affirmative answer to the above ques- 
tions should be of great interest to edu- 
cators, particularly if the means for 
achieving these goals is both practical 
and inexpensive. Subliminal stimuli ad- 
ministered while the subjects are view- 
ing a motion picture, television broad- 
cast, or filmstrip may provide the de- 
sired affirmative. 

By way of definition, subliminal 
means that which is below the percep- 
tible threshold. The threshold is the 
level at which things become perceiv- 
able (understood through the senses). 
The working threshold varies from per- 
son to person and, within the individual, 





* Dr. Oxhandler, until recently assistant 
professor of audiovisual education at Penn- 
sylvania State University, has been appointed 
associate director for research in the Audio- 
Visual Center at Syracuse University. 


from minute to minute. Subliminal 
stimuli may take the form of: 

1. Sounds, either above or below the 
normal auditory range of 16 to 16,000 
cycles. 

2. Sounds heard but not associated 
with the main focus of attention, such 
as noises which stimulate or touch off 
subconscious overtones, or sounds which 
trigger psychological moods or feelings. 

3. Visual imagery which is seen pe- 
ripherally (not the main focus of atten- 
tion) and which acts as a trigger for 
past responses. 

4. Visual symbols projected con- 
stantly at an impoverished light inten- 
sity below the normal visual threshold. 

5. Visual stimulation through high- 
speed projection at a rate faster than 
the eye can outwardly comprehend. 

The hypothesis tested in this study 
is that subliminal stimuli administered 
while a film is being projected will pro- 
duce significant differences in the re- 
sponses of students taking the test as 
opposed to responses of students not 
subliminally stimulated. Populations 
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were drawn from undergraduate stu- 
dents taking courses in audiovisual edu- 
cation at Pennsylvania State University 
and at Fresno State College, California. 

Prior to this experiment a review of 
pertinent literature and research was 
conducted. Examples of the more signi- 
ficant studies in visual stimulation are 
mentioned here: 

1. In 1934, Publicity Films, Ltd., 
London (1), conducted an experiment 
in which the letters “OXO” were flashed 
and superimposed over a portion of 
blank film projected on a screen. The 
one-minute film was projected 20 times 
at One minute intervals to an audience 
of eight people, who were requested to 
record any impressions received from 
the experience. At the end of the per- 
formance, two people wrote down the 
letters OXO and one other person wrote 
OOOH. The experiment was repeated 
several times producing similar results 
each time. 


2. Several seasons ago two groups, 
working independently with greatly di- 
vergent methods, claimed that they 
could increase the sales of products and 
influence the attitudes of subjects 
through mass subliminal stimuli. James 
Vicary (2), of Subliminal Projection, 
Inc., conducted a series of tests in mo- 
tion picture theaters in which he dem- 
onstrated an appreciable jump in the 
sales of a soft drink and of popcorn. 
Mr. Vicary was said to have flashed his 
subliminal messages during the feature 
film at a speed in excess of 1/2000th 
of a second. Becker and Corrigan (3), 
working out of New Orleans, developed 
a similar method of projection which 
they have used over both motion pic- 
tures and television with similar claims 
of success. 


3. Dr. George Guthrie, who is pro- 
fessor of psychology at Pennsylvania 
State University, and several of his col- 
leagues have conducted studies in which 
individual subjects were electric-shock 
conditioned to produce a physical skin 
reaction (as measured by a galvanom- 
eter) to several out of a large sam- 
pling of nonsense syllables. Upon com- 
pletion of the conditioning, subjects 
were exposed to the same nonsense syl- 
lables, this time projected under impov- 
erished light conditions. The GSR 
(Galvanic Skin Response), however, 
remained the same even though the 
subject could not outwardly see the 
visual stimulus. 


In an attempt to apply these findings 
to the solution of some of the questions 
posed in the opening paragraph, a 
small pilot experiment was designed by 
Thurston M. Reeves, Ben C. Duke, and 
Eugene Oxhandler at Pennsylvania 
State University. An excerpt from the 
motion picture The Informer was used. 
Mr. Duke devised two sets of test ques- 
tions, the first set open ended; the sec- 
ond, multiple choice. The two key ques- 
tions dealt with information that was 
not in the film itself, namely: “In what 
city did the action take place?” and 
“What was the date of the action?” 

It was decided that the place, arbi- 
trarily designated as CORK, would be 
subliminally induced by impoverished 
direct, continuous, illumination. A 2x2 
piece of darkened plastic was used and 
the letters CORK were scratched into 
the surface. The slide was placed in a 
150-watt projector which, in turn, was 
connected to a voltage transformer, al- 
lowing the illumination to be impover- 
ished below the level of projected visi- 
bility. The date, NOV. 4, was similarly 
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incised on a 3% x4 slide and placed in 
the appropriate projector. The bulb and 
reflector were removed from the hous- 
ing of the projector and a small elec- 
tronic flash outfit was substituted. The 
electronic flash produced an image on 
the screen every five seconds for the 
14-minute duration of the film. The 
flash itself lasted approximately 1/750th 
of a second. 

At the close of the first showing of 
the film, the open-ended test was ad- 
ministered to the 16 viewers. No view- 
ers were able to identify the place as 
CORK; however, three people indi- 
cated the date as NOVEMBER and one 
person had the date as NOV. 4. The 
film was then reshown, this time with 
only CORK on the screen. At the con- 
clusion of the second showing the mul- 
tiple choice test was given with no con- 
clusive evidence in favor of CORK. 
The three responses of NOVEMBER 
were repeated in the results of the mul- 
tiple choice examination. Some of the 
subjects indicated that they thought they 
could see the message even though the 
investigators were unable to do so. 

As a result of this pilot investigation 
the following experiment was under- 
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taken. The film Lessons From The Air, 
produced by the British Information 
Service, was selected because it was 
used regularly as part of the audiovisual 
education courses given both at Penn 
State and at Fresno State College and 
because it has a sequence in which 
teachers hand in evaluation cards on 
radio programs they have reviewed. On 
one of these cards the date 20/4/43 
appears, which might indicate a date 
slightly prior to the actual film produc- 
tion time. 

During the winter of 1958 a test, 
shown as Figure I, was devised and 
given to approximately 250 students 
regularly registered in audiovisual edu- 
cation courses at Pennsylvania State 
University and to some 250 students at 
Fresno State College, all after they had 
viewed the film. The students were told 
that their instructors wished to check 
the effectiveness of the film in com- 
municating both facts and ideas. One 
hundred tests from each group were 
randomly selected and scored. Table 1 
records the results of these tests. It 
is readily apparent that both questions 
11 and 16 (“Lessons From The Air was 
made at a date shown in the film. What 











TABLE 1 

Penn State Fresno Total Penn State Fresno Total 
No. (100) (100) (200) || No. (100) (100) (200) 
l 76 59 135 11 0 0 0 
2 33 11 44 12 30 16 46 
3 89 80 169 13 43 31 74 
4 69 26 95 14 34 27 61 
5 92 82 174 : 15 90 86 176 
6 67 45 112 16 0 0 0 
7 56 5 61 | 17 95 74 169 
8 91 79 170 | 18 71 40 111 
) 76 54 130 | 19 65 23 88 
10 81 6 87 || 20 29 36 65 











QUIZ: LESSONS FROM THE AIR 


ANSWER QUESTIONS ON THIS SHEET IN THE SPACES PROVIDED. 


oOo ND 


15. 
16. 


17. 


18. 
19. 


20. 


. What type of school does the educational wing of the BBC cover? All types— 


elementary and secondary; rural and urban. 


. The Chinese narrator’s name was Chou Chen. 


What was used to create the sound of boats in water? Brick in pan of water. 


. Who controls educational broadcasting in Britain? A committee of educators and 


radio directors. 


5. Are teachers represented in #4? Yes 


The Chinese broadcast began at /0 A.M. 
The Celeste was substituted for the piano in the Chinese program. 


Coconut shells were used to create the sound of horses’ hooves in the rhythms 
broadcast. 


. Stories from history were done in dramatic form. 
. What universities or colleges were mentioned? Oxford, Corpus Christi. 


. “Lessons From The Air” was made at a date shown in the film. What was the 


date? April 20, 1943. 


. The film lists one major use of educational radio in Great Britain. Name the use. 


To bring reality to the classroom. 


. How far ahead are the programs planned? 6 months. 


. Who were the three planners of the China program? Geographer, Chinese expert, 


radio director. 
Did the Chinese expert agree with the first script? No. 
Ann Driver teaches rhythm activities to children over the BBC. 


How do parents use the BBC with their children? They listen to the same programs 
and discuss them with the children. 


The children are asked to imagine by the narrator of the historical drama. 
How does the radio bring maps to life. By creating images of far-off places. 


Who does the criticism of BBC educational broadcasts? Selected committees of 
teachers. 


Note: The correct answers appear in italics. 


Ficur_E I 
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was the date?” and “ teaches 
rhythm activities to children over the 
BBC.”) failed to get correct answers 
from anyone. In the case of question 16, 
however, ambiguous phrasing precluded 
an accurate answer. A broader inter- 
pretation in scoring to include such an- 
swers as “a music teacher,” “rhythms 
teacher,” etc., would have produced a 
normal response. In item number 11 
no one came even close to the correct 
answer. 

During the 1959 summer session at 
Pennsylvania State University three 
groups of students designated here as 
groups X, Y, and Z were shown the 
same film and given the same test. As 
the film was projected a second ma- 
chine, a 344x4 slide projector with the 
lamp and reflector removed and a Hei- 
land II Strobonar flash lamp substituted 
in the housing, was used to flash the 
missing date on the screen at ten-second 
intervals. A 3%4x4 black-coated lan- 
tern slide with the date APRIL 20, 
1949 scratched into its surface and 
filtered down with a piece of greyed 
photographic film, was used to produce 
the subliminal message. The date was 
deliberately changed to eliminate the 
chance of someone seeing the actual 
date on the film and answering the ques- 
tion correctly. The Heiland II Strobonar 
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produced a high intensity flash at 
1/2000th of a second. The message was 
focused on the screen with unfiltered 
illumination in a completely darkened 
room. The filter was inserted when the 
slide was used during the experiment. 
After the showing it was discovered 
that the focus of the slide had inad- 
vertently been disturbed prior to the 
film showing to group X, quite prob- 
ably prejudicing the results for that 
group. 

Table 2 indicates the test results for 
groups X, Y, Z, as well as a T test of 
Proportion between the subliminally- 
stimulated and non-stimulated groups. 
In group X, with 28 subjects, we find 
two partially correct scores producing 
a non-significant T of 1.23. Group Y, 
with 16 subjects, had one completely 
correct answer and six partial answers 
for a T of 3.44, significant at the 1 per- 
cent level. Group Z, with 26 subjects, 
had seven partial answers for a T of 
2.66, also significant at the 1 percent 
level. 

On all the other questions, groups 
X, Y, and Z showed T scores of no 
significance indicating that all groups 
answered the questions equally well. 
The results definitely show that the 
subliminal stimulus made a distinct im- 
pression. 





TABLE 2. 

Group April April20.~— April 1949 «1949 April 20,1949 “T” 
ey cig 1 1 1.23 
(28) 
ee 5 1 3.44 
(16) 

- 5 3 , 2.66 


(26) 
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Most certainly the conditions for this 
experiment leave much to be desired, 
particularly from a scientific standpoint. 
Below are several factors which might 
tend to make the results questionable: 


© Students reported seeing something 
on the screen during the film showing. 
One student indicated that he could see 
part of the message. 


e The extra projector and operator, 
no matter how well disguised, were ele- 
ments of distraction. 


¢ No attempt was made to predict 
accurately a common threshold in terms 
of illumination, frequency of stimulus, 
speed of exposure. 


e No attempt was made to assess 
the subject’s physiological and psy- 
chological backgrounds. (Several stu- 
dents reported dreaming about the date, 
others stated that for a few days after 
exposure to the subliminal message, the 
date would come to them whenever 
they closed their eyes.) 


e Randomization was _ haphazard 
rather than deliberate. Randomiza- 
tion might well be necessary for more 
accurate results. 


¢ Other variations of subliminal stim- 
uli were not utilized. No attempt was 
made to (a) cue the viewer to look for 
a particular fact, idea or concept within 
the film itself, and (b) reinforce exist- 
ing facts, ideas or concepts to increase 
learning where an item on a test has 
shown weak or inconclusive responses. 


To facilitate a more accurate study 
of the phenomenon of subliminal per- 
ception and its relation to the teaching- 
learning process, some of the following 
items will need to be provided: 


1. A device (now in the preliminary 
design stage—no funds available for its 
construction) to allow experimenters to 
control the major variables of (a) in- 
tensity of illumination from 0 watts to 
at least 2000 watts, (b) duration of 
the flash from 1/10th of a second to 
1/10,000th of a second, and (c) fre- 
quency of the flash from one every min- 
ute to at least 24 per second. (A source 
of illumination for such a controlled 
device seems, at present, to be the main 
drawback to its construction.) 


2. Adequate records and interview 
tapes on all subjects tested. 


3. Definite follow-up procedures to 
check duration of learning and other 
subconscious effects and to discover 
whether the message delivered appears 
in some people at a later date and in 
another context. 


4. Room conditions permitting pro- 
jection entirely without student aware- 
ness. 


Exploration of the potentials of sub- 
liminal stimulation might well be at- 
tempted in all of the areas suggested 
in the opening paragraph (sounds as 
well as visual imagery) to truly deter- 
mine the possibilities of this technique. 
Certainly the findings of this simple 
study indicate that there are affirmative 
answers to these questions. 
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A Brief Study of 


Televised Laboratory Instruction 


WARREN F. SEIBERT AND 
JURGEN M. HoniG 


This article is a condensed and modi- 
fied version of a February 1959 mimeo- 
graphed report by the authors. The 
mimeographed original includes copies 
of measuring devices used in gathering 
data for the study which can be had by 
writing the senior author at the Audio- 
Visual Center at Purdue University. Dr. 
Seibert, who was recently on leave to 
the U. S. Office of Education as research 
coordinator for the Educational Media 
(Title VII) Program, returns to Purdue 
July 1 as TV Program Research Con- 


| in possible educational uses 
of television has increased at a fairly rap- 
id rate during the past half dozen years 
and evidence of this interest can easily 
be found in the many popular articles 
on the subject, and in the increasing 
number of research reports that deal 
with the results of these instructional 
trials. Controversies concerning the 


sultant. Dr. Honig is currently with the 
Lincoln Laboratories, Massachusetts In- 
stitute of Technology. The writers 
thank Frederick Vratney, Post-Doctoral 
Fellow in Chemistry; D. F. Kasten, 
graduate student in Psychology; the 
staff of the Purdue TV Production Unit; 
and the several graduate students who 
served as performance evaluators. All 
made contributions to the study on which 
this report is based. It goes without say- 
ing that the student subjects were essen- 
tial and their cooperation appreciated. 


place of television in formal education 
have been by no means absent; never- 
theless, the writers are naive enough to 
hope that these disputes can be dimin- 
ished in intensity or frequency through 
increased reference to the findings from 
careful research and through reliance on 
the most substantial accumulated expe- 
rience. Perhaps the following report 
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will make a contribution in the desired 
direction. 

Reports such as those from the Penn- 
sylvania State University (2, 3), the 
State University of lowa (1, 11), Miami 
University (8, 9), and Purdue Univer- 
sity (4, 10) provide a basis for re- 
marks such as Kumata’s that “On the 
whole, television students have done as 
well as other students .. .” (6, p. 5). 
Although a considerable number of care- 
fully planned, well executed studies 
lend support to the Kumata statement 
and although there is no intention of 
discounting the significance of the “no 
significant difference” statements con- 
tained in earlier reports, it is true that 
the great majority of these studies have 
utilized as criteria the students’ factual 
knowledge, ability to apply the prin- 
ciples learned, or “information gain.” 
Relatively little attention has been given 
to the difficulties or effectiveness of tele- 
vised instruction in laboratory skills. 


In 1956 and again in 1957 The Fund 
for the Advancement of Education 
made grants to the Purdue University 
Department of Chemistry to permit 
study of televised and other audiovisual 
instruction in the general chemistry lab- 
oratory. Valuable experience was gained 
in the course of these instructional 
trials and, for the most part, indications 
were that televised instruction in the 
general chemistry laboratory at the col- 
lege level can offer several useful solu- 
tions to instructional problems which 
exist or can be anticipated.’ During the 
two years of preliminary trial, some in- 
formation concerning student laboratory 


‘The most recent and inclusive of the 
reports concerning televised and other audio- 
visual instruction in the general chemistry 
laboratory is entitled The Utilization of 
Audio-Visual Aids in the General Chemistry 





achievement was gathered and exam- 
ined. This information has been helpful 
and interesting but its quality has been 
limited by the necessities and pressures 
which arise when several hundred stu- 
dents are enrolled in a single course. 


Purpose of the Study 


This study was planned and conducted 
so that information might be derived to 
supplement experiences and information 
from the earlier and more extended 
trials of televised laboratory instruction 
at Purdue University. In comparison 
with earlier trials, it was judged that 
this study would lose extensiveness but 
would compensate for the loss through 
a significant increase in the precision of 
variable control. Attention has been 
given to maintaining instructional con- 
ditions typical of those present in earlier 
trials and typical of those which might be 
used in the future. As in most previous 
studies of televised instruction, the prin- 
cipal question to be answered was: Can 
televised (laboratory) instruction serve 
as an effective substitute for instruc- 
tion as normally given? 


Procedures of the Study 


The study plan included provision 
for data from 60 subjects (S s)—12 
S s in each of five groups—but experi- 
ence has shown that, in spite of all pre- 
cautions, some attrition of the sample 
is inevitable. Therefore, 70 S s were 
signed to participate and a plan was 
developed to eliminate excess data, if 
any, randomly. The volunteer S s were 
obtained through the Department of 


Laboratory Work at Purdue University, Au- 
gust 1958. Copies of this and the two earlier 
reports are available on request from the De- 
partment of Chemistry, Purdue University, 
Lafayette, Indiana. 
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Psychology and the professor in charge 
of its introductory course.” 

Each of the 70 S s was randomly as- 
signed to one of the following five in- 
struction-testing conditions: 


1. Televised laboratory instruction 
with criterion measurements taken 
within about two hours of the comple- 
tion of the instructional presentation. 


2. Televised laboratory instruction 
with criterion measurements taken dur- 
ing a session held five weeks after the 
instructional presentation was given. 


3. Conventional laboratory instruc- 
tion with criterion measurements taken 
within about two hours of the comple- 
tion of the instructional presentation. 


4. Conventional laboratory instruc- 
tion with criterion measurements taken 
during a session held five weeks after 
the instructional presentation had been 
given. 

5. No laboratory instruction—crite- 
rion measurements gathered from one 
random half of this group on the eve- 
ning during which instructional presen- 
tations were given. Measurements were 
gathered from the second random half 
of the group during the session held five 
weeks later. 


Instructional presentations to which 
the S s were exposed were based on the 
contents of two sound films prepared 
by the Department of Chemistry within 
the project supported by The Fund for 


“There were 12 S s in the initial group 
of 70 volunteers who had extensive previous 
chemistry training and were judged not to be 
suitable for this study. They were eliminated 
from the list of participants and substitutes 
were obtained prior to the random assignment 
of S s to instruction-testing conditions. “Ran- 
dom,” as applied here and elsewhere in this 
paper, refers to the use of random numbers 
tables in determining conditions applicable to 
each participant or prospective participant. 


the Advancement of Education. The 
film contents were supplemented some- 
what by the two instructors who pre- 
pared and gave the instructional presen- 
tations.* One film, with a running time 
of about six and a half minutes, covers 
techniques of mass determination and 
the various types of balances; it also 
gives particular attention to the use of 
the triple beam balance. The other, 
with a running time of about eight and 
a half minutes, covers the apparatus 
and procedures for making volumetric 
measurements. 


The first session with the S s was 
devoted to a brief and necessarily vague 
introduction to the study, to pretesting, 
to the instructional presentations, and 
to the gathering of criterion data from 
one random half of the group of §S s. 
In the introduction, S s were told that 
patience and reasonable efforts were 
expected of them, that they should not 
communicate with others concerning 
participation in the study, and that each 
would receive an explanatory report on 
the study after the data had geen gath- 
ered and examined. Pretesting consisted 
of gathering responses to the customary 
personal data items and the administra- 
tion of a vocabulary test. 


Following the pretest, S s were di- 
vided into three groups as determined 


* The “instructor variable” presents a uni- 
versal problem in studies such as this one and 
steps were taken to minimize its effects. The 
two instructors were familiar with the pur- 
poses and procedures of the study and par- 
ticularly familiar with the nature of the cri- 
terion data to be gathered. They were in close 
contact as each prepared his presentation. In 
addition, both hold the Ph.D. degree in physi- 
cal chemistry and have had teaching experi- 
ence suited to the tasks performed. In the 
opinion of the writers, these considerations 
make it unlikely that instructor differences 
contributed significantly to the outcomes of 
the study. 
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by a previously prepared random as- 
signment plan. These groups were com- 
posed of those S s who would receive 
televised instruction, those who would 
receive conventional instruction, and 
those who would receive no instruction. 
Each group was assigned to one of three 
adjacent rooms. Care was taken to pre- 
vent sound transmission between the 
rooms during instructional presenta- 
tions. The two presentations lasted 20 
to 30 minutes, the televised presenta- 
tion perhaps five mintues longer than 
the other. Following this, the random 
assignment plan was again used to iden- 
tify and release those S s whose criterion 
data were to be gathered at a later date. 
An 18-item test covering material from 
the presentations was administered to 
the remaining S s. Twelve were four- 
alternative multiple choice items, and 
the rest were matching items. 

The last activity in this first session 
with the S s was to gather laboratory 
performance evaluations. Each S was 
required to carry out a relatively simple 
titration while performance of each was 
evaluated by an assigned graduate stu- 
dent. All evaluators were volunteers 
who had been briefed concerning the 
pitfalls and specific duties of the evalu- 
ator. Immediate, accurate recording 
and a pleasant but noncommittal atti- 
tude toward the S s were particularly 
stressed. The comments and evaluations 
were recorded on specially prepared, 
itemized scoring sheets and no evalu- 
ator * received advance information con- 
cerning the group to which any given 


‘The one exception to this was that the 
instructor who presented “conventional in- 
struction” also served as a second, or du- 
plicate, evaluator of five S s’ performance. 
Evaluations he supplied were used only in 
estimating the rater-to-rater reliability of 
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S belonged. Also, since only eight S s 
could be evaluated simultaneously, their 
order of appearance for laboratory 
evaluations was made dependent upon 
a previously prepared random order 
plan. 

Five weeks after the first session with 
the S s, the second and last session 
was devoted to the same testing and 
evaluation activities as had been in- 
volved in the latter part of the earlier 
session. The S s who reported for this 
session were those who had been ran- 
domly assigned to the “delayed measure- 
ment” condition. There were also “non- 
instructed” S s who were reporting for 
the first time and the first few minutes 
were therefore devoted to introducing 
them to the study procedures and gath- 
ering their responses to personal data- 
vocabulary test items. 


Tests and Performance Evaluations 


test 


The personal data-vocabulary 
form was prepared to permit a descrip- 
tion of the sample and to provide the 
covariate for subsequent analyses of 
criterion data. The vocabulary test in- 
cluded 24 four-choice, multiple-choice 


items; its «Auder-Richardson Formula 
20 reliability (7, p. 586 ff.) was found 
to be .740. The test was scored with 
unit weights for each item and no cor- 
rection for wrong responses. 


The 18-item knowledge test, one of 
the two criterion measures used in the 
study, was developed from the two 
film scripts and pretested twice in in- 
troductory chemistry classes. Prelim- 


evaluations, rather than as principal indexes 
of performance skill. The cther of the sec- 
ond, or duplicate, evaluators was the tele- 
vision instructor but he, of course, had not 
seen S s in “his” group and could not iden- 
tify them by other means. 
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inary forms of the test were item ana- 
lyzed after both pretestings and items 
were deleted or rewritten on the basis 
of information gathered. All items ap- 
peared to be functioning satisfactorily 
in the second pretesting and only minor 
modifications were introduced in the 
preparation of the test form used in this 
study. Scoring involved unit weights for 
each item and no correction for wrong 
responses. Data gathered from the 60 
S s in the study indicated that this test 
had a Kuder-Richardson Formula 20 
reliability of .672. 

The laboratory performance evalua- 
tions were included to provide the S s 
with a task which had been described 
and demonstrated during the earlier 
presentations and to measure skills in 
executing the task. Evaluators’ scoring 
sheets included a listing of the 14 pro- 
cedural steps to be followed by the S s, 
a listing of appropriate subtopics within 
each step, and indications of the credit 
points to be granted for any level or 
type of performance. Thus, the evalu- 
ators were primarily concerned with 
observing closely, matching observed 
performance with the most appropriate 
performance description listed on the 
scoring sheet, and recording any com- 
ments that seemed to be required. 

Each § received a performance out- 
line at least several minutes prior to 
appearance in the laboratory and this 
outline listed the 15 main steps to be 
followed in executing the titration; it 
did not specify details of procedure. 
Evaluators were specifically instructed 
not to assist § s, either directly or by 
subtle cues, except when the observed 
poor performance would nullify the ade- 
quacy of S’s subsequent efforts. Evalu- 
ators were instructed to give no credit 
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for poor performance that required cor- 
rection. To check the reliability of these 
evaluations, duplicate evaluations were 
obtained for an unselected sample of 
ten S s. The two chemistry instructors 
who had given the instructional presen- 
tations made these evaluations and, al- 
though there was some information ex- 
changed between evaluators scoring the 
same performance, the majority of the 
judgments was rendered independently. 
The rank order correlation between the 
sets of paired evaluations was .888. 


Preliminary Findings 


Attrition of the student sample dur- 
ing the course of the study did not prove 
to be a problem of serious proportions 
since all but six of the 70 S s appeared 
and participated; these six included rep- 
resentatives from four of the five groups. 
Excess data were randomly eliminated 
as planned. The S s whose data are 
included in the following analyses had 
a mean age of 20.48 years and an age 
range from 18 to 44. They included 
31 female and 29 male students from 
the schools of engineering, agriculture, 
home economics, science, education, and 
the humanities. 


Vocabulary pretest scores from the 
five groups of S s were analyzed by 
means of a single classification analysis 
of variance (5, p. 315ff.) as a check 
on the initial intergroup comparability. 
The mean vocabulary pretest scores from 
the groups were as follows: TV-imme- 
diate measurement, 15.17; TV-delayed 
measurement, 16.00; conventional-im- 
mediate measurement, 14.75; conven- 
tional-delayed measurement, 16.08; and 
no instruction, 16.67. The F-ratio from 
the analysis of variance was .453 (with 
4 and 55 df.); differences among the 
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means are not statistically significant 
and the groups can be considered com- 
parable. 


Results of the Study 


Single classification analysis of co- 
variance (12, p. 387ff.) was used to 
treat data from the two criterion, or 
learning, measures. Vocabulary pre- 
test scores served as the covariate in 
both of these analyses. Summaries of 
the results are presented in Tables 1 
through 4. 

Both the initial and the adjusted 
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mean scores shown in Table 1 suggest 
that which is demonstrated by the F- 
ratio in Table 2; test performance dif- 
ferences among the five groups were not 
Statistically significant. It is apparent 
that, although the 18-item knowledge 
test had face validity and evolved out 
of seemingly less “polished” versions, 
it was inadequate when employed as a 
discriminating criterion of learning in 
the present sample. More evidence on 
this subject is presented later. 
Laboratory performance mean scores 
for each of the five groups are shown 


TABLE !|—INITIAL AND ADJUSTED CHEMISTRY KNOWLEDGE TEST 
MEANS FOR THE FIVE GROUPS (N=12, EACH GROUP) 


Television 


Immediate Delayed 
Mean Score 10.167 12.417 
Adj. Mean 10.236 12.384 


Conventional No 
Immediate Delayed Instruction 
10.500 12.250 10.750 
10.620 12.207 10.636 


TABLE 2—SUMMARY OF COVARIANCE ANALYSIS INVOLVING 
CHEMISTRY KNOWLEDGE TEST SCORES 


Source SS df 
Treatments 47.7261 4 
Error 467.2467 54 

58 


Total 514.9728 


11.9315 1.379 >.05 
6527 


TABLE 3—INITIAL AND ADJUSTED LABORATORY PERFORMANCE 
MEANS FOR THE FIVE GROUPS (N=12, EACH GROUP) 


Television 


Immediate Delayed 
Mean Score 61.000 56.750 
Adj. Mean 61.420 56.552 


Conventional No 
Immediate Delayed Instruction 
60.250 60.250 50.417 
60.919 59.991 49.725 


TABLE 4—SUMMARY OF COVARIANCE ANALYSIS INVOLVING 
LABORATORY PERFORMANCE SCORES 


Source ss. df 
Treatments 1112.3487 4 
Error 4895.1866 54 
Total 6007.5353 58 





V F p 
278.0871 3.068 <.05 
90.6516 
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in Table 3 and a summary of the cor- 
responding analysis of covariance is 
shown in Table 4. Information from 
these two tables gives adequate indica- 
tion of the significant performance dif- 
ferences existing among the five groups. 
The F-ratio reported in Table 4 (3.068, 
with 4 and 54 df.) indicates significance 
somewhat beyond the .05 level of con- 
fidence. A review of Table 3 will show 
a relatively tight grouping of means 
from the four instructed groups but a 
noticeable gap between them and the 
mean from the non-instructed group. 
The difference between the adjusted 
means from the two delayed measure- 
ment groups was tested with the appro- 
priate f-test and the obtained ¢ was 
.884, considerably less than required for 
significance at the .05 level of confi- 
dence. 

Further treatment of the data was 
prompted by the fact that both in- 
structors emphasized certain portions of 
the presented material, perhaps at the 
expense of other portions on which 
S s were later tested and evaluated. Be- 
cause of this it was suspected that cer- 
tain items and steps in the two criterion 
measures would not effectively discrim- 
inate between instructed and non-in- 
structed S s. Therefore, each instructor 
was asked to check a copy of the knowl- 
edge test and the laboratory perform- 
ance scoring sheet and to indicate the 
items and steps which had, in his opin- 
ion, received attention in the instruc- 
tional presentation he had given. Indi- 
cations received from the instructors 
and item-by-item (or step-by-step) tab- 
ulations of the results from each of the 
five groups are presented in Tables 5 
and 6. 


Information in Table 5 gives further 


indication of the inadequacies of the 
18-item knowledge test. Only items 2, 
8, and 16 provided even the mildest 
indications of their ability to discrim- 
inate between instructed and non-in- 
structed groups and these item “suc- 
cesses” are too minute to be regarded 
as reliable. It can again be seen that 
the attempt to develop a reasonably 
valid and sensitive measure of acquired 
knowledge was, to say the least, un- 
satisfactory. 


Information in Table 6 is more con- 
sistent than that in Table 5 and shows, 
among other things, that the perform- 
ance level of all instructed groups was 
above that of the non-instructed group 
on eight of the 14 performance steps. 
Step-by-step comparisons of the summed 
performance of television- and conven- 
tionally-taught groups show seven steps 
(nos. 3, 4, 7, 9, 10, 11, and 12) on 
which television S s were slightly supe- 
rior, five steps (nos. 1, 2, 6, 13, and 14) 
on which conventionally-taught S s were 
slightly superior, and two steps (nos. 5 
and 8) on which the differently-taught 
groups performed equally well. 


Instructors’ indications of content 
coverage, as shown in Table 6, provide 
information of interest, even though not 
in complete accord with the measured 
levels of laboratory performance. For 
example, step 3 gives indication of dis- 
agreement between instructors’ state- 
ments and measured performance; the 
Table shows that only the convention- 
ally-taught S s received instruction ap- 
propriate to this step, yet the television 
S s showed some superiority in execut- 
ing the step. Also, on step 9, only the 
television instructor indicated (partial) 
coverage of appropriate material, yet all 
instructed groups executed this step 





TABLE 5—INSTRUCTORS’ INDICATIONS OF CONTENT COVERAGE AND 
POINTS EARNED BY EACH GROUP ON EACH ITEM 
IN THE CHEMISTRY KNOWLEDGE TEST 


Test Material Covered in: Television Conventional No 
Item TV Conventional | Immed. Delay Immed. Delay Instr. | Total 
l 10 8 5 8 6 37 
2 yes yes 10 12 11 10 9 52 
3 yes 3 7 3 5 7 25 
4 yes yes l 4 2 5 6 18 
5 yes yes 4 6 7 7 6 30 
6 yes yes 6 11 7 11 10 45 
7 yes yes 11 10 10 12 10 53 
8 11 9 12 9 8 49 
9 yes yes 2 4 5 5 3 19 
10 yes yes 2 9 4 7 2 24 
11 7 10 7 11 7 42 
12 yes yes 11 10 10 11 11 53 
13 yes 0 a 3 5 5 16 
14 yes 12 11 10 12 12 57 
15 9 11 8 7 9 44 
16 yes 4 3 5 3 0 15 
17 10 11 9 11 9 50 
18 9 10 8 8 9 44 
All 11 9 122 149 126 147 129 673 


TABLE 6—INSTRUCTORS’ INDICATIONS OF CONTENT COVERAGE AND 
POINTS EARNED BY EACH GROUP ON EACH STEP OF 
THE LABORATORY PERFORMANCE EVALUATION 


Lab. Material Covered in: Television Conventional No 
Step TV Conventional | Immed. Delay Immed. Delay Instr. | Total 
l 98 88 101 100 100 | 487 
2 yes yes 102 103 122 111 87 525 
3 yes 51 56 i 47 45 | 243 
4 ? 34 34 30 33 27 158 
5 34 34 33 35 33 169 
6 60 51 60 54 57 282 
7 yes 34 34 31 30 24 153 
8 ? 21 22 21 22 16 102 
9 ? 78 68 60 70 47 323 
10 yes yes 63 53 57 55 51 279 
11 20 12 18 13 13 76 
12 36 33 33 33 33 168 
13 yes yes 57 49 63 70 39 278 
14 yes yes 44 44 50 50 33 221 
All 8* a 732 681 723 723 605 3464 


* The eight steps reportedly covered during the televised presentation include three 
steps which, by instructor indications, had received limited or incomplete coverage. 
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more adequately than the non-instructed 
S s. On the positive side, those steps 
which reportedly received adequate cov- 
erage in both presentations (nos. 2, 10, 
13, and 14) were among the steps which 
apparently discriminated between per- 
formance of instructed and non-in- 
structed § s. 


Conclusions 


In this brief study, all reasonable 
efforts were made to gain control over 
extraneous variables while gathering 
limited but valid information concern- 
ing the relative effectiveness of televised 
and conventional instruction in labora- 
tory skills. There are unanswered ques- 
tions concerning the methods, the ef- 
fects, and the meanings which still out- 
number the few questions which may 
have been answered but these should 
not obscure what was attempted and 
found. In brief, the results indicate 
that one of the two criteria employed 
in the study was little more than worth- 
less. The other criterion measure gave 
evidence that, although the instruction 
produced measureable effects, it appar- 
ently made no difference whether groups 
were taught by conventional means or 
by means of television. Statistically, 
these instructed groups performed 
equally when executing their laboratory 
tasks. This evidence is not contradicted 
by related information from the earlier 
and more extended trials of televised 
laboratory instruction. 

Perhaps, at best, the inclusion of “de- 
layed measurement” groups has added 
some small amount of respectability to 
this study. Their inclusion certainly has 
not guaranteed that unequivocal an- 
swers can be given to those questions 


which deal with the “long-term” effects 
of the two forms of instruction. Since 
these “long-term” effects are extremely 
important, it is hoped that they will re- 
ceive increasing attention in the future. 
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Student Attitudes Toward 


Closed-Circuit Instructional Television 


HOWARD M. BOBREN AND 
SHELDON L. SIEGEL * 


A PPROXIMATELY 37 colleges and uni- 
versities are currently utilizing closed- 
circuit television; 22 are studying its 
possibilities for instruction.' Questions 
arise as to the relative effectiveness of 
television and traditional teaching tech- 
niques, and attitudes toward the former. 
This paper reports a study of student 
attitudes toward a course partially pre- 
sented on closed-circuit television. 


METHOD 


An engineering course offered at the 
University of Illinois during the fall 
semester of 1958 and designed to fa- 
miliarize students with the resistance of 
materials was utilized for the study. The 


National Education Association, Ameri- 
can Association of Colleges for Teacher Edu- 
cation. Closed-Circuit Television in AACTE 
Member Institutions. Committee on Studies 
and Subcommittee on Television and Teacher 
Education, Bulletin Vol. 9, No. 5, May 16, 
1958. 


course traditionally met once a week 
for a two-hour period which was usually 
consumed by a _ lecture-demonstration, 
short discussion, and quiz. Approxi- 
mately 12 students, meeting in a labora- 
tory with an instructor, were accommo- 
dated in each section. The laboratory 
equipment, used for demonstration pur- 
poses only, was not handled by the 
students. 

Although over 100 students take this 
course each semester, it has been taught 
in small laboratory sections, not only 
because most of the equipment cannot 
be moved to larger lecture halls, but 
also because these larger rooms do not 


* Howard M. Bobren is assistant to the 
director, Institute of Communications Re- 
search and Sheldon L. Siegel is research 
assistant, College of Education, both at the 
University of Illinois. 

The authors wish to thank the Depart- 
ment of Theoretical and Applied Mechanics 
and the Institute of Communications Re- 
search for their cooperation in this study. 
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give the students an adequate view of 
the demonstrations. Therefore, this 
course was well suited for teaching via 
television. The showing of laboratory 
equipment to a large body of students 
at one time prevented costly duplication 
of running the electrically driven ma- 
chinery, and the television camera was 
able to present to every student a large, 
clearly defined picture of occurrences. 
During the first semester in which the 
course was taught via closed-circuit tele- 
vision it was offered both over television 
and in the traditional manner. This of- 
fered an opportunity to compare results. 

General plan of the experiment. Ten 
sections were selected for the experi- 
ment, five of which were taught in the 
conventional manner and five via tele- 
vision. All sections contained about 12 
students. The conventional sections 
were taught as in the past, meeting 
weekly for a two-hour period of lecture- 
demonstration, discussion, and quiz. 
Each was taught by a different instruc- 
tor. The television sections also met 
once a week for a two-hour period. 
For the lecture-demonstration, all stu- 
dents assembled in a single large room 
equipped for television viewing. Either 
at the end of the televised lecture, or 
at some other time during the two-hour 
period, the group would break up into 
sections of about 12 each for discussion 
periods, weekly quizzes, or other exam- 
inations. Each television section had its 
own instructor who took his turn pre- 
paring and delivering the televised 
lectures. 


Subjects. All subjects were male—52 
in the experimental group and 60 in the 
control group. Grade-point averages of 
the two groups did not differ signifi- 
cantly. When the students registered 
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for the course at the beginning of the 
semester, they did not know whether 
they would be in a television or con- 
ventional section. All foreign and some 
transfer students were eliminated from 
the study because of difficulty in ob- 
taining certain necessary variables. 

Test Material. At the end of the 
course all students in the experiment 
were administered a questionnaire de- 
signed to measure the attitudes of the 
students toward the course and the way 
it was taught. A special form of the 
Semantic Differential * was developed. 
Subjects were asked to rate the follow- 
ing concepts: “Teaching where the in- 
structor is in the classroom,” “Teaching 
where the instructor is on television,” 
“This course,” and “Your instructor(s) 
in this course.” The following scales 
were used: worthless-valuable, un- 
pleasant-pleasant, clear-hazy, unfair- 
fair, good-bad, successful-unsuccessful, 
meaningful-meaningless,  wise-foolish, 
confusing-understandable, boring-inter- 
esting, and simple-complicated. A 
sample page of the Semantic Differential 
is shown here: 


THIS COURSE 


worthless— :—:—:—:—:—:—valuable 
unpleasant —:—:—:—:—;—:— pleasant 
Gee..: $$ tata hazy 
ee —fair 
ee 
successful_:—:—:—:—:—:~-unsuccessful 
meaningful_—:—:—:—:—:—:—meaningless 
See 
confusing—:—:—:~-:—:—:—understandable 
boring —:—:—:—:—:—:~— interesting 
simple—:—:—:—:—:—:—complicated 


* Osgood, C. E.; Suci, G. J.; and Tannen- 
baum, P. H. The Measurement of Meaning. 
Urbana, Illinois: The University of Illinois 
Press, 1957. 
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In addition, the following questions 
were asked: 


How well did you understand the pur- 
pose of laboratory experiments in this 
course? 


By watching experiments, how clearly 
did you perceive the physical relation 
between controls, dials, and the speci- 
men? 

Were you confused about these or 
was the over-all picture clear? 


How efficiently was the laboratory 
time utilized? 


How well, do you think, will you be 
able to describe what you learned in 
this course six months from now? 


How clearly did the course bring out 
the relation between physical behavior 
in the laboratory and theoretical for- 
mulations developed? 


In making a general comparison with 
other engineering courses you have 
taken, how would you rate this 
course? 


On the average, how much time per 
week did you spend on this course 
outside of class? 


If you have the option next semester 
of enrolling in either a TV section or 
a conventional section of this course 
and if both sections are taught by the 
same instructor (whom you like) and 
are given at desirable hours which fit 
in with your schedule, which section 
will you choose? 

The following information was also 
obtained for each subject in the experi- 
ment: cumulative university grade-point 
average; mid-term examination grade; 
and final course grade. 

Administration of test materials. Be- 
cause we wanted the frank opinions of 
the students, we were concerned that 
their answers not be motivated by the 
fact that they had not yet received their 
final grades. Therefore we assured them 
that their responses would in no way 
affect their grades. To put them even 
more at ease, one student in each sec- 
tion, selected at random, collected all 


copies of the questionnaire and held 
them until the final grades had been 
posted. 


ANALYSIS AND RESULT 


Means and standard deviations were 
computed on each item in the question- 
naire for both the experimental and the 
control group, and the two groups were 
compared on each item. In addition, 
mid-term and final grades were com- 
puted and compared. 

With respect to the first objective of 
this study—evaluating the relative effec- 
tiveness of closed-circuit television 
teaching as compared to conventional 
teaching—no differences were found. 
Neither the mid-term examination grade 
nor the final course grade differed sig- 
nificantly between the two groups. 

However, the examination of the 
acceptability of closed-circuit television 
showed striking differences. When asked 
to rate the concept “This course,” the 
two groups differed significantly on 10 
out of the 11 scales. The television 
group rated the course less pleasant 
(p=.001), less clear (p=.001), less 
fair (p=.01), less good (p=.01), less 
successful (p=.001), less meaningful 
(p=.05), less wise (p=.01), less 
understandable (p= .001), less inter- 
esting (p= .05), and less simple (p= 
.O5), than did the conventional group. 
The only scale that did not differentiate 
between the two groups was worthless- 
valuable. 


When asked to rate the concept 
“Your instructor(s) in this course,” the 
two groups differed significantly on eight 
out of the 11 scales. The television 
group rated their instructors as being 
less valuable (p=—.001), less clear (p= 
O01), less fair (p=.001), less good (p 
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—.05), less successful (p=.001), less 
meaningful (p=.01), less wise (p= 
.001), and less interesting (p=—.05), 
than did the conventional group. The 
scales that did not differentiate signifi- 
cantly between the groups were: con- 
fusing-understandable, simple -compli- 
cated, and unpleasant-pleasant. 

When asked to rate the concept 
“Teaching where the instructor is on 
television,” the two groups differed sig- 
nificantly on only one scale; the tele- 
vision group considered it less under- 
standable (p=—.01), than did the con- 
ventional group. There were no signi- 
ficant differences between the groups on 
the concept “Teaching where the in- 
structor is in the classroom.” 

The questions concerning the course 
itself showed similar results as_ indi- 
cated by the following: 

How well did you understand the pur- 


pose of laboratory experiments in this 
course? 


The television group stated that they 
understood less clearly (p=.001) 
than did the conventional group. 


By watching the experiment, how 
clearly did you perceive the physical 
relation between controls, dials, and 
the specimen? Were you confused 
about these or was the overall picture 
clear? 


The television group stated that it was 
less clear (p=.001) than did the con- 
ventional group. 


How clearly did the course bring out 
the relation between physical behavior 
in the laboratory and theoretical for- 
mulations developed? 


The television group stated that the 
relation was less obvious (p=.05). 
How well, do you think, will you be 
able to describe what you learned in 
this course six months from now? 


The television group stated that they 
would do less well (p=.05). 


In making a general comparison with 
other engineering courses you have 


taken, how would you rate this 
course? 


The television group rated the course 
poorer (p=.001). 

The groups did not differ on the 
question “How efficiently was the la- 
boratory time utilized?” The television 
group stated that on the average they 
spent more time on the course outside 
of class than did the conventional group 
(p=.01). Sixty-five percent of the stu- 
dents in the television section stated 
that they would prefer a conventional 
section were they to take the course 
over, 27 percent would take a television 
section, and 8 percent expressed no 
choice. Eighty-four percent of the stu- 
dents in the conventional section stated 
that they would prefer a conventional 
section were they to take the course 
over, 8 percent would take a television 
section, and 8 percent expressed no 
choice. 

We also examined the relationship 
between students’ university grade-point 
averages and the question pertaining to 
preferences. The better the student the 
more likely he was to prefer a con- 
ventional section (p=.05). 


DISCUSSION 


When we consider the first objective 
of this study (the relative effectiveness 
of closed-circuit television compared to 
conventional teaching), it appears that 
there is no difference. The study failed 
to show differences in performance when 
measured by a mid-term examination 
or a final course grade. 

When we consider the second objec- 
tive of this study (the acceptability to 
the students of closed-circuit television 
teaching) the results are clear. The 
Pennsylvania State University Studies 
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(using other techniques) reported that 
students’ general attitudes toward tele- 
vised instruction as compared with di- 
rect instruction were “mainly neutral or 
slightly negative,” but they found no sig- 
nificant differences. Our results clearly 
show that television students have rela- 
tively negative attitudes toward the 
course and the instructors of the course 
as compared to the students in the con- 
ventional sections. 

With respect to the relative efficiency 
of the two methods of teaching, it is 
noted that the students in the televised 
sections stated that they spent more time 
on the course outside of class than did 


‘Carpenter, C. R.; Greenhill, L. P.; and 
others. An Investigation of Closed-Circuit 
Television for Teaching University Courses. 
Instructional Television Research Project No. 
1, Instructional Film Research Program, July 
31, 1955. University Park: Pennsylvania State 
University, p. 1. 


the students in the conventional sec- 
tions, but their performances on the 
examinations were the same. 


SUMMARY 


The objectives of this study were to 
examine (1) the relative effectiveness 
of closed-circuit television for class- 
room teaching compared to conventional 
teaching and (2) the acceptability of 
closed-circuit television teaching to the 
students. A course was offered both by 
closed-circuit television and by conven- 
tional teaching. Students within each 
group were asked to rate the course and 
other related concepts. In addition, their 
examination grades were obtained. No 
differences were found with respect to 
performance. The students taught via 
television rated the course and their in- 
structors relatively lower than those who 
received conventional instruction. 
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REPORT ON THE ADMINISTRATION OF 
EDUCATIONAL FILM ACTIVITIES IN 14 COUNTRIES 


PART II 


A Survey Made for UNESCO by the 


International Council for Educational Films 


SEE the Fall 1959 issue for Part I which 
comments generally on the 14 countries 
treated individually here. The first half 
of the survey reveals that film industries 
in most of the 14 countries undertake the 
production of educational films, but in 
some countries, production is either en- 
tirely governmental or governmentally 
sponsored or undertaken by private pro- 
ducers. The distribution of educational 
films is often handled by a government 
agency for more effective centralization, 
less expense, and better quality of materials 
offered to schools. Local financing of edu- 
cational film activities follow the general 
principles of the school system in each 
country—either it is municipal (tax sup- 
ported) or it is private with different de- 
grees of municipal support. To help solve 
the problem of insufficient teacher training 
facilities, some government institutions, 
universities, and film centers have or- 
ganized inservice training for teachers, 
and are now offering courses for special 
certificates in the audiovisual field. 


Austria 


This country presents perhaps the 
most typical example of a pyramid of 
administration, developed from the cen- 


tral level downwards. Institutions on all 
three levels are operated by the federal 
government and the province (Lander) 
governments. 


The nucleus of the Austrian system 
is an institution under the Federal Min- 
istry of Education, the Federal Center 
for Picture and Educational Film (Bun- 
desstaatliche Haupstelle fur Lichtbild 
und Bildungsfilm). This federal center 
forms the roof of a country-wide organ- 
ization, consisting of nine provincial 
centers (Landesbildstellen) and 92 local 
centers (Lokale Bildstellen). The admin- 
istrative costs for all these centers are 
covered by a special federal budget of 
about $120,000. 


The production of educational films 
for schools is centralized at the federal 
center, and prints are deposited with the 
provincial and local centers. Production 
and printing costs are covered by a 
special budget of about $80,000 result- 
ing from pupils’ contributions which are 
compulsory and are higher in secondary 
schools than in elementary schools. 
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Provincial centers produce some simple 
films of local educational interest. The 
yearly release of new titles from the 
federal center is increasing. Seven new 
titles were released in 1950, 24 titles in 
1955. There is a marked dominance of 
film imports over home production as 
well as of films for secondary schools 
over films for elementary schools. The 
number of films available to schools has 
grown from some 380 titles (28,000 
prints) in 1950 to some 500 titles 
(50,000 prints) in 1955. All films are 
silent and black-and-white. There is no 
tendency to change to sound or color. 
It is estimated that some 80 prints of a 
film for secondary schools and 120 
prints of a film for elementary schools 
are hardly sufficient to meet the demand. 

The schools have at their disposal 
approximately 3900 16mm film _pro- 
jectors, of which some 3850 are silent 
and only 50 sound. The purchase of all 
projectors is channeled through the fed- 
eral center. Municipalities buy projec- 
tors for elementary schools; the federal 
government for secondary schools. The 
films are supposed to be shown in class- 
rooms that are equipped with facilities 
for darkening. Daylight projection is not 
practiced. 

Apart from the leaders of the local 
film centers, there are no film super- 
visors or coordinators in the municipali- 
ties or schools. 


Training in the use of audiovisual 
media is compulsory in teachers colleges, 
with from four to 12 hours devoted to 
this subject. The training includes peda- 
gogic methods, projection technique, 
and classroom practice. Inservice teach- 
ers may get further training in courses 
arranged by the film centers. Film 
expression and film art is taught in 


noncompulsory courses for inservice 
teachers. 


Belgium 


The pyramid of administration has 
developed from the central level down- 
wards. A regional level does not exist. 

A special bureau of the Ministry of 
Education (Le Service Cinematograph- 
ique du Ministere de I’Instruction Pub- 
lique) is responsible for production and 
distribution of educational films for 
schools, working on a budget of about 
$290,000. For the time being it rarely 
produces films but concentrates on im- 
port. 

Sixty-two titles were released in 1950, 
65 titles in 1955. 

The number of films available to 
schools has grown from some 690 titles 
(1900 prints) in 1950 to some 1610 
titles (8550 prints) in 1955. There is a 
clear tendency towards a preference for 
sound films as well as towards the intro- 
duction of color films. 


It is calculated that some 12 prints of 
a film are sufficient to cover the demand. 

The governmental center distributes 
the films free of charge to the schools, 
which do not receive any other subsidy, 
either provincial or municipal, for their 
film activities. 

The schools have at their disposal 
approximately 2700 16mm film pro- 
jectors, out of which some 200 are silent 
and 2500 sound. State schools are given 
projectors free of charge from the gov- 
ernmental center; other schools do not 
receive any support for the purchase of 
projectors. In new schools the films are 
mostly shown in special screening rooms. 
Otherwise, classrooms are used. All 
such localities are equipped with dark- 
ening facilities, and daylight projection 
is not practiced. 
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Normally, one of the teachers in a 
school acts as a coordinator for film 
activities. Contrary to the school librar- 
ian, he is, however, not specially re- 
munerated for this work. 

No compulsory teacher training in the 
use of audiovisual media exists, and 
film expression and art is not taught. 
Certain audiovisual courses for inservice 
teachers are given by a Catholic organi- 
zation (the CEDOC). 


Canada 


There is no country-wide administra- 
tive pyramid. Education is a respon- 
sibility of the province, and in each 
province the pyramid has developed 
from the regional level downwards. 

A federal institution for the produc- 
tion and distribution of films exists (the 
National Film Board of Canada), but 
only a very small part of its activities 
is devoted to films for schools. Nor- 
mally, not more than two or three edu- 
cational films for schools are produced 
per year. These are produced under 
the guidance of a special subcommittee 
of the Canadian Education Association, 
a cooperative body representing the 
provincial school authorities. 

Otherwise, the production and cen- 
tral distribution of educational films is 
undertaken by private enterprise. Home 
production is insignificant, and films are 
imported predominantly from the United 
States, the United Kingdom, and France. 
It has been impossible to calculate the 
number of new film titles released in 
the country each year. 

In order to cover the demand, ap- 
proximately 50 to 75 prints of a film 
for elementary schools and about five 
to 20 prints of a film for secondary 
schools are found to be necessary. 


In each province, the school authori- 
ties maintain an educational film library. 
The number of film titles in these li- 
braries vary from 500 to 2500, and the 
number of prints from 1000 to 6000. 
Practically all are sound films, and the 
percentage of color films is rapidly in- 
creasing. In two provinces there is a 
censorship on educational films; in some 
others the provincial authorities advise 
the schools on the quality of films to be 
used. 


About 500 local film libraries exist, 
administered by the municipal school 
authorities. They are in various ways 
and to varying degrees supported by the 
provincial authorities. The bigger li- 
braries are complete audiovisual centers 
in Toronto and Vancouver. All libraries 
distribute their films to the schools at 
expedition cost. Some universities also 
distribute films to schools. The schools 
have at their disposal approximately 
6000 16mm film projectors. Nearly all 
are sound projectors; silent projectors 
are no longer bought. Projectors are 
purchased by the municipal school au- 
thorities, which in some provinces re- 
ceive a governmental subsidy covering 
50 percent of the costs. Many rural 
schools make use of projectors that cir- 
culate with the National Film Board’s 
traveling units for showing films that 
they borrow from local or provincial 
libraries. The films are preferably 
shown in darkened classrooms, although 
in most schools only a few classrooms 
are equipped with darkening facilities. 
Daylight projection is not popular. 


In many parts of the country, there 
are coordinators of audiovisual aids 
either for a school or a school district. 
They are teachers, and their position is 
rather similar to that of school librarians. 








They are not specially remunerated but 
are, at least in larger districts, allowed 
a certain amount of hours off for their 
work. 

Although teacher training in the use 
of audiovisual aids is not compulsory, 
most institutions give such training. Ele- 
mentary school teachers normally re- 
ceive more training at teachers colleges 
than secondary school teachers receive 
at the universities. Inservice teachers 
get further training in courses organized 
by coordinators or municipal and pro- 
vincial authorities. In Ontario, a yearly 
course for 60 teachers leads to qualifi- 
cation for posts as coordinators. Film 
expression and art is covered only when 
there is special interest on the part of 
the instructor. 


Denmark 


The pyramid of administration has 
developed from the central level down- 
wards with governmental institutions at 
central and regional levels. 

Production is undertaken by the Dan- 
ish Cultural Films (Dansk Kulturfilm), 
a private institution, governmentally 
supported and controlled. Only a minor 
part of its $60,000 budget is, however, 
devoted to educational films for schools. 

The Danish Government Film Office 
(Statens Filmcentral) is responsible for 
film import and distribution, and works 
with a budget of about $150,000. Its 
activities include adult education, chil- 
dren’s entertainment films, et cetera. 

About 60 new film titles are released 
¢ach year by the governmental center. 
Most of the films are imported. Some 
220 titles were available to schools in 
1950, some 450 titles in 1955. Most of 
the films are silent and black-and-white, 
but the percentage of sound and color 
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films is increasing. In order to cover the 
demand, some 20 prints from a silent 
film and some 10 prints from a sound 
film are normally needed. 


The films are distributed directly 
from the governmental center to the 
schools, and a number of prints are de- 
posited in a revional branch office in 
order to facilitale transport. The 
schools get the films either on a yearly 
subscription or under a rental system. 
A number of local film libraries have 
recently been created, based on a 10 
years’ print lease system. Prices are 
kept very close to the expedition cost 
level. Municipal school authorities 
cover the local costs for film supply. 

The schools have at their disposal 
approximately 1800 16mm film _pro- 
jectors, of which some 1500 are silent 
and some 300 sound. The number of 
sound projectors is increasing. The pro- 
jectors are bought by the municipal 
school authorities, which can receive a 
governmental subsidy for this purpose. 
In new schools, the films are shown 
most often in an auditorium with dark- 
ening facilities. In older schools, nor- 
mally one classroom may be darkened 
and used as a film room. Daylight pro- 
jection is not practiced. 

Film or audiovisual coordinators or 
supervisors do not exist, except in the 
city of Copenhagen where there is one 
such post, comparable to that of an 
assistant headmaster. 

Teacher training in the use of films is 
compulsory. It is arranged by courses 
given with the assistance of the govern- 
mental film center. In such courses, 
groups of 30 future teachers are given 
a six hours’ all-round training. Courses, 
lectures, et cetera, for inservice teachers 
are also given free of charge by the 
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governmental center at the request of 
schools or teachers groups. Film ex- 
pression and art is not covered by such 
courses, but literature on these subjects 
is available to the participants. 


France 


The pyramid of administration has 
developed both from the central level 
downwards and from the local level up- 
wards. Governmental institutions exist 
on the central and regional levels, but 
other regional institutions are financed 
and administered by provincial or local 
school authorities. 


Under the Ministry of Education, a 
special institution—-the National Peda- 
gogic Institute (l'Institut Pedagogique 
National)—is responsible for the supply 
of educational films to schools. It is 
guided in this respect by the Govern- 
mental Committee for Educational Films 
(Commission Ministerielle du Cinema 
d’Enseignement). Production is carried 
out approximately as follows: 


One-third by the audiovisual center 
in St. Cloud (Centre Audio-Visuel de 
Ecole Normale Superieure de _ St. 
Cloud), the official production unit of 
the Ministry of Education, which pro- 
duces mainly experimental, pedagogical 
and scientific films. It is a pilot unit. 

One-third by private companies under 
contract signed with the Institut Peda- 
gogique National. 

One-third by private companies, on 
scenarios approved by the Commission 
Ministerielle du Cinema d’Enseigne- 
ment, under contract signed with trade 
unions or private industrial concerns and 
sometimes with subsidies from the IPN. 

The Centre Audio-Visuel de St. Cloud 
also makes French versions of imported 
foreign films, re-edits film footage for 


educational versions, and helps univer- 
sities and teachers in local productions. 


The total price of these productions, 
as well as the sponsorship of certain pri- 
vate productions, is covered by a budget 
of some $350,000. The figures given 
above cover only the production made 
under the sponsorship of the Ministry of 
Education. Approximately 100 other ed- 
ucational films are produced every year 
by French private companies on their 
own or under the sponsorship of other 
ministries or private concerns. 


The yearly release of new film titles 
through the libraries has increased be- 
tween 1950 and 1955 from 20 titles to 
over 100. About one quarter of these re- 
leases are imported, and they are almost 
equally divided between elementary and 
secondary school levels. Production of 
color films is increasing, although some 
color titles’ are distributed also in black- 
and-white versions because of the lower 
printing costs. The number of educa- 
tional films available to schools has in- 
creased from some 160 titles in 1950 to 
some 400 titles in 1955. It is calculated 
that to cover the demand some 150-200 
prints are needed from films for elemen- 
tary schools and some 30-50 prints from 
films for secondary schools. 


The National Institute maintains a 
central film library, mainly intended to 
serve secondary schools. It also finances 
and administers 17 regional film li- 
braries, intended to serve both secondary 
and elementary schools. The films in 
these libraries are available to all public 
schools on a subscription basis, and 
subscribing secondary schools receive a 
special subsidy from the Ministry of 
Education for this purpose. Prints are 
also sold to 30 provincial film libraries 
(Cinematheques de Departements), 
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mainly intended to serve elementary 
schools. The National Institute pays the 
staff salaries for these provincial film 
libraries, but their film budget is carried 
by provincial and local school authori- 
ties. Finally, prints are sold to a num- 
ber of local film libraries in large cities 
and important schools, financed by the 
local school authorities. It should be 
noted, however, that private schools are 
allowed neither to subscribe to regional 
film library programs, nor to purchase 
prints of governmental films. 


Schools have approximately 8000 
16mm film projectors at their disposal. 
Very few silent projectors are used. For 
the purchase of projectors, schools re- 
ceive various types of governmental sub- 
sidies. Generally, films are shown in 
the classrooms, but in big schools spe- 
cifically equipped film rooms often exist. 
Room darkening is preferred to daylight 
projection. 


There are generally no film supervisors 
or coordinators in the schools. For sec- 
ondary schools, however, a system of 
National Institute “correspondents” has 
been planned, but this system is not yet 
functioning. 


Teacher training in the use of visual 
aids is compulsory, but due to lack of 
competent instructors many teachers 
have not received this training. The 
audiovisual center arranges, in coopera- 
tion with various institutions and organi- 
zations, regional courses of from two to 
three days and central courses of from 
eight to 15 days, both for future and in- 
service teachers. Certain central courses 
of a considerably more extensive nature 
are also given each year. In all central 
courses, certain elementary production 
techniques are taught, and several hours 
are always devoted to film expression 


and art. The creation of teachers and 
youth film societies (cine-clubs) is 
promoted. 


The Federal Republic of Germany 


The pyramid of administration, one of 
the most effective among the existing 
systems, has developed from the regional 
level both upwards and downwards. The 
educational authorities of the provinces 
(Lander) carry the budgets of both the 
central and regional institutions. 

The Educational Film Institute (In- 
stitut fur Film und Bild in Wissenschaft 
und Unterricht) in Munich functions as 
the central body for production and dis- 
tribution. It is not supported by the fed- 
eral government, but its budget is carried 
cooperatively by the Lander with yearly 
subscriptions based on the number of 
pupils in the schools. The Institute pro- 
duces films of its own, but most of the 
releases are films bought from other 
producers in and outside the country. 

Thirteen new titles were released in 
1950, and 63 titles in 1955. The total 
number of films available to schools was 
350 in 1950, and 520 in 1955. Most 
of the films are silent and black-and- 
white, but the number of sound films is 
increasing, and color films have recently 
been introduced. Some 600 prints of 
a film for elementary schools may be 
needed to cover the demand. 


Prints of the films are sold by the 
Institute at prices slightly over printing 
costs to the 13 province film centers 
(Landesbildstellen) or channeled through 
them to more than 500 regional (Kreis- 
bildstellen) or municipal (Stadtbild- 
stellen) centers administered by local 
school authorities. All these centers func- 
tion as complete audiovisual service in- 
stitutions, and the province film centers 
can also undertake certain simple pro- 
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ductions of local interest. The centers 
distribute the films free of charge to 
schools. 


The schools have at their disposal 
approximately 22,000 16mm film pro- 
jectors, of which some 20,000 are silent 
and some 2000 sound. There is, how- 
ever, a marked tendency towards the 
purchase of sound projectors. Local 
school authorities pay for the projectors, 
but they receive subsidies for this pur- 
pose from the Lander governments 
through the province film centers. Films 
are mostly shown in classrooms with 
darkening facilities, but some new 
schools have special film rooms. Day- 
light projection is not practiced in ordi- 
nary schools (it exists, however, in some 
vocational schools). 


There are no coordinators in the 
schools or school districts, but the leader 
of the local or regional film center is 
usually a teacher. He is not specially 
remunerated for this work but his ex- 
penses are covered. The province film 
centers are equipped with fulltime staff. 

Teacher training in the use of audio- 
visual aids is not compulsory, but all 
film centers are very active in the pro- 
motion of effective film usage by ar- 
ranging courses, conferences, and lec- 
tures for all categories of teachers. 
Teachers take a growing interest in film 
expression and art, and the Educational 
Film Institute endeavors to present ma- 
terial for such studies. Some films have 
been produced for this purpose. 


Italy 


A new law, regulating educational film 
activities, has been expected for several 
years. In the meantime, the situation 
seems somewhat unclear. The pyramid 
of administration has developed from 


the central level downwards, but recently 
more weight has been put on the re- 
gional level, and in reality local efforts 
play a more essential part than may be 
expected from the organization plan. 


Within the Ministry of Education, a 
special institution for educational films 
exists for both production and distribu- 
tion (Cineteca Scolastica). Because of 
an insufficient budget—some $30,000 
per year—its production has, however, 
been severely restricted during past 
years. Because of this, a number of pri- 
vate producers have greatly increased 
their release of educational films. 


In 1955, close to 50 new titles were 
released and the number of films avail- 
able to schools was brought up to some 
150 titles. Most of these titles are black- 
and-white sound films, but color films 
have also been introduced. To cover 
the demand from the elementary schools, 
some 100 prints of a film may be needed. 


The films are rented to schools, and 
Cineteca Scolastica maintains a subscrip- 
tion system. Prints are also sold to 
provincial film libraries, which are now 
being transformed into complete audio- 
visual centers (Centri Provinciali per 
i Mezzi Audiovisivi nell’Educazione). 
These centers are controlled by the pro- 
vincial school authorities and financed 
by pupil subscriptions. According to 
the organization plan, 92 such centers 
exist, but their stages of development 
vary widely. 

The schools have at their disposal 
approximately 5000 16mm film projec- 
tors, of which some 1500 are silent and 
some 3500 sound. There is a marked 
tendency towards the purchase of sound 
projectors. In principle, municipal school 
authorities are supposed to buy the 
projectors, but in many cases parent 
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organizations cover the costs. Films are 
generally shown in darkened classrooms. 

There are no film supervisors or co- 
ordinators in the schools. The leaders 
of provincial centers are always head- 
masters, and they are partly relieved of 
other duties. 

Teacher training in the use of audio- 
visual aids exists here and there on local 
initiative, but there is no general sched- 
ule for such activities. 


Luxemburg 


It would hardly be appropriate to 
speak of a pyramid of administration in 
a country like Luxemburg as parallel 
to, for instance, France, the Federal 
Republic of Germany, or the United 
States. The administrative structure of 
the centralized system in Luxemburg 
corresponds more to that of a regional 
center in some of the larger countries. 

A special bureau of the Ministry of 
Education (L’Office du Film Scolaire) 
is responsible for the supply of educa- 
tional films to schools and also for pro- 
duction of slides, so it may be considered 
as an audiovisual center. It is working 
with a budget of about $10,000. 

About 350 film titles with some 4000 
prints are available to schools. Since 
1950, there has been a tendency towards 
the use of sound films. 

The governmental center distributes 
the films free of charge directly to all 
schools in the country. 

The schools have at their disposal ap- 
proximately 150 16mm film projectors, 
out of which some 110 are silent and a 
little over 40 sound. The purchase of 
projectors is a responsibility of the mu- 
nicipal school authorities, and they do 
not receive any governmental support 
for this purpose. 


Teacher training in the use of films 
is compulsory. It is given by the teachers 
colleges and includes both the technical 
and pedagogical aspects of the subject. 


The Netherlands 


The pyramid of administration has 
developed from the central level down- 
wards. A regional level does not exist. 


There is one central institution for the 
production and distribution of films to 
schools, the Netherlands Educational 
Film Foundation (Stichting Nederlandse 
Onderwijs Film). This institution is 
owned and administered by the three 
organized school systems of the coun- 
try (the Catholic, Protestant, and non- 
conformist schools) and is government- 
ally subsidized. 

Ten new film titles were released by 
the central institution in 1950; 21 titles 
in 1955. The number of film titles avail- 
able to schools has grown from some 
80 in 1950 to some 160 in 1955. Home 
productions are predominantly 16mm, 
silent, black-and-white and generally in- 
tended for elementary schools. Most of 
the imported films are sound, black-and- 
white, and: intended for secondary 
schools. Color films have been intro- 
duced. In order to meet the demand, 
some 65 prints are needed of a silent film 
for elementary schools and some 15 
prints of silent or sound films for sec- 
ondary schools. 

The films are distributed to schools 
under a subscription system based on a 
fixed contribution per pupil per year in 
the subscribing schools. The pupils 
themselves are not supposed to pay. At 
a slightly increased subscription fee, 
schools may also borrow silent projec- 
tors from the central institution. 


Approximately 2750 16mm film pro- 
jectors of which some 2600 are silent 
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and 150 sound, are owned by schools, 
and 470 silent projectors, owned by the 
central institution, are circulated to 1450 
schools. Thus the total number of pro- 
jectors used by the schools amounts to 
more than 3200. There is a growing 
municipal interest in providing schools 
with projectors. No marked tendency 
is visible as to any change in the pro- 
portion of silent and sound projectors. 
Films are shown in darkened classrooms 
or special film rooms. Daylight pro- 
jection is not popular. 

Audiovisual supervisors or coordina- 
tors do not exist in each school but only 
where several schools share one projec- 
tor. He is always a teacher but is not 
specially remunerated for this work. 

Out of 82 teachers colleges in the 
country, 22 have taken up audiovisual 
training in cooperation with the central 
institution. Methodology is taught by 
local teachers, and technical instruction 
is given by the central institution. Class- 
room practice culminates in a test, which 
is controlled by the central institution 
who gives a certificate to the trainee. 

A new act, promulgated this year, 
makes audiovisual teacher training com- 
pulsory, and has also created greater 
interest to include the study of film ex- 
pression and art in such programs. There 
is no organized instruction given by the 
central institution. 


Norway 


This country presents the most typical 
example of a complete pyramid of ad- 
ministration, developed from the local 
level upwards. Institutions on all three 
levels are operated directly or indirectly 
by the municipal authorities. 

The central institution for production 
and distribution of educational films for 


ordinary schools acts as a branch of the 
Film Center of the Municipalities (Kom- 
munenes Filmcentral), a joint organiza- 
tion for the municipally owned cinemas. 
A governmental film center exists also 
(Statens Filmcentral), responsible for 
supplying films to adult education, voca- 
tional training and certain specialized 
schools. There is close cooperation be- 
tween the two institutions. A separate 
committee, acting as an advisory board 
to the Ministry of Education, grants 
educational visas for films to be used in 
schools. There is close cooperation be- 
tween all three bodies. 


There were 98 film titles granted edu- 
tional visas in 1950 and 238 titles in 
1955. Imports dominate, and the pro- 
portion of home productions is between 
10 and 15 percent of the total released. 
Some 300 film titles were available to 
schools in 1950, some 800 titles in 1955. 
There is a more and more marked domi- 
nance of sound films, and the proportion 
of color films is increasing. Ten prints 
of a film are normally sufficient to cover 
the demand, but sometimes this figure 
is considerably exceeded. 


The educational film activities of the 
Kommunenes Filmcentral are financed 
from revenues on the commercial (cin- 
ema) side of the organization. The films 
are either rented or sold to schools or 
to regional and local film libraries fi- 
nanced and organized by the municipali- 
ties with the aid of government subsidies; 
only printing and expedition costs are 
charged. 


The schools have at their disposal 
approximately 3100 16mm projectors, of 
which some 600 are silent and some 
2500 sound. Purchase costs are carried 
by the municipal school authorities, 
which normally receive a 50 percent 
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subsidy from country authorities for this 
purpose. In new schools there are usu- 
ally special film rooms. Most of the 
classrooms have no darkening facilities. 
Daylight projection is not practiced. 


Although no standard system exists, 
there is normally one teacher who acts 
as film coordinator in each school. He 
is not specially remunerated for this 
work. 


Teacher training in the use of audio- 
visual media has been compulsory since 
1950, but training institutions are free to 
establish their own standards. Generally 
two hours are devoted to methodology 
and technical instruction. Classroom 
practice is given to future elementary 
school teachers. The Ministry of Edu- 
cation advisory committee arranges each 
year a three weeks’ course for inservice 
teachers. Film expression and art are 
not included in such study programs. 


Sweden 


The pyramid of administration has 
developed from the local level upwards. 
On the central level there is no institu- 
tion administered or supported by the 
authorities. 


The production and central distribu- 
tion of educational films for schools is 
undertaken by private enterprise, and 
several producers are active in this field. 
They do not receive any subsidies from 
authorities. Only a small number of films 
for teacher training have been partly 
sponsored by the Royal Board of Edu- 
cation. 

Generally, some 40-50 new film titles 
are released per year, and imported 
films outnumber the home productions. 
Elementary schools enjoy a far richer 
supply than secondary schools, due to 
difference in demand. In total, some 
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1000 film titles are available to schools, 
and the Royal Board of Education edits 
an advisory catalog of films revised by 
evaluation groups. The previous domi- 
nance of silent films is rapidly changing 
into a marked dominance of sound films. 
The proportion of color films is growing. 
Some 100 prints of a film normally cover 
the demand of the elementary schools, 
whereas some 20 prints are mostly suf- 
ficient for the secondary schools. 


The distribution of films to elemen- 
tary (municipal) schools is normally 
done by selling prints to some 100 re- 
gional and 200 local film libraries 
administered and financed by municipal 
school authorities. Some regional film 
libraries receive subsidies from pro- 
vincial authorities. Secondary (state) 
schools normally rent films directly from 
the producers. 


The schools have at their disposal 
approximately 6000 16mm film projec- 
tors, about half silent and half sound. 
For the past couple of years, schools 
have bought only sound projectors. 
Local school authorities do not receive 
any special governmental subsidy for the 
purchase of projectors. In principle, 
films are shown in classrooms but very 
often also in natural science rooms or, 
in new schools, in special film rooms. 
Only rooms with darkening facilities are 
used, and daylight projection is not 
practiced. 


Where there is no local film library, 
there are no special film supervisors or 
coordinators in schools or school dis- 
tricts. Almost all leaders of regional 
and local film libraries are teachers. 
They receive a restricted remuneration 
for this work but, contrary to school 
librarians, they are not relieved of any 
other duties. 
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Teacher training in the use of audio- 
visual aids is compulsory, but standards 
vary. Some five hours are normally de- 
voted to methodology and _ technical 
instruction. Classroom practice is always 
given. Week-end courses and some 
longer courses are given each year for 
inservice teachers, arranged by the Royal 
Board of Education, teachers organiza- 
tions, school inspectors and regional film 
libraries, often in cooperation with the 
producers. Film expression and art are 
normally not included in the study pro- 
grams, but the interest in this subject 
is growing. 


Switzerland 


Education in this country is a respon- 
sibility of the provinces (cantons). The 
pyramid of administration has thus de- 
veloped from the regional level upwards 
and downwards, but the production side 
is weak. 

There is no central institution for pro- 
duction of educational films for schools. 
Productions are private and mainly 
sponsored by political, religious, or com- 
mercial organizations and sometimes by 
private commercial companies. Some 
films are, however, produced by the 
Association of Swiss Educational Film 
Centers (Vereinigung Schweizerischer 
Unterrichtsfilmstellen) and by regional 
film centers. 


Some 50 new film titles, mostly im- 
ported, are released each year, and 400 
film titles are available to schools through 
the film centers. The films are pre- 
dominantly silent and black-and-white, 
but the number of sound and color films 
is increasing. At present, five prints of 
a film are sufficient to cover the demand. 


Four regional film centers exist, two 
of them cantonal and two intercantonal. 


They are either administered or financed 
and controlled by the canton govern- 
ments. The cooperative body of these 
four centers (the VESU) is officially 
recognized by the Conference of Direc- 
tors of all 25 cantonal Departments of 
Education. The two intercantonal cen- 
ters distribute their films on a rental or 
subscription basis, the two cantonal cen- 
ters distribute free of charge. Contribu- 
tions from canton governments vary 
widely, although they are never high 
(some canton governments do not con- 
tribute at all). Also municipal film 
budgets vary from nothing at all to in- 
significant amounts. This has led to 
a situation in which, although no contri- 
butions are in principle expected from 
the pupils, the most obvious budget gaps 
are sometimes filled by private sponsor- 
ship or collections. 


The schools have at their disposal 
approximately 2000 16mm film projec- 
tors, of which some 1800 are silent and 
200 sound. During the last few years, 
there has been a tendency in favor of 
sound. In some cantons, the municipali- 
ties may receive a governmental subsidy 
of 10 to 70 percent for the purchase of 
projectors. The films are generally shown 
in darkened classrooms or special rooms. 
Daylight projection is not practiced. 


Only in some schools does a teacher 
function as film coordinator. He is then 
not specially remunerated for this work, 
but his position is similar to that of the 
school librarian. 


Teacher training in the use of audio- 
visual aids is not compulsory but exists 
in several teachers colleges where some 
four to six hours are devoted to a gen- 
eral training. Local courses for inservice 
teachers are often arranged and in some 
cantons the film centers arrange more 
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extensive courses. Film expression and 
art are not included in the program. 


United Kingdom: England and Wales 


The pyramid of administration has 
developed from the local level upwards 
with some governmental support to an 
institution at the central level. 

Two official institutions exist and 
work closely together with some staff in 
common. The National Committee for 
Visual Aids in Education has been es- 
tablished by the local education authori- 
ties and teachers associations. Its field 
of activity is educational policy. The 
Educational Foundation for Visual Aids 
(EFVA) is financed by the local educa- 
tion authorities but it has also received 
certain governmental subsidies. Its main 
activities are distribution and production. 
Distribution and production are also 
undertaken by commercial organizations. 


No statistics are kept of the number 
of educational films for schools released 
each year, but in 1955 about 1700 titles 
from producers at home and overseas 
were available to schools through the 
EFVA. Most of these are sound films, 
and the number of color films is rapidly 
increasing. Some 30 prints of a film are 
normally sufficient to cover the demand. 


Films are distributed in two ways. 
Some films are rented directly to schools, 
the rental paid for by the local education 
authorities. Prints are sold to regional 
or local film libraries, financed by the 
local education authorities, who distrib- 
ute them free of charge to the schools 
in the area. 


The schools have approximately 8000 
16mm film projectors at their disposal 
(mostly sound projectors). The pur- 


chase of projectors is financed by the 
local education 


authorities. The films 
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are mostly shown in darkened class- 
rooms. Daylight rear projection is also 
widely practiced, though more with 
filmstrips than with films. 

In some cases, the local education 
authorities have created a special post 
of “visual aids officer” to cover their 
area. In other cases, a teacher acts as 
coordinator without any special remun- 
eration. Sometimes there is no coordi- 
nator at all. 

Teacher training in the use of films 
is increasing in various forms, although 
it is not compulsory. Inservice teachers 
may receive training in courses arranged 
partly by local education authorities, 
colleges and universities, partly by some 
200 associations called “Teacher’s Vis- 
ual Aid Groups.” There is also a coun- 
try-wide teachers association, with re- 
gional and local branches, that occupies 
itself with film expression and are under 
the term “film appreciation.” This move- 
ment cooperates very closely with the 
British Film Institute. 


United Kingdom: Scotland 


The pyramid of administration has 
developed from the local level upwards. 
Central and local institutions are gov- 
ernmentally supported. ~A regional level 
does not exist. 

Production of educational films for 
schools is undertaken by the Joint Pro- 
duction Committee of the Scottish Edu- 
cational Film Association, and distribu- 
tion by the Scottish Central Film 
Library. Both institutions receive sub- 
sidies for these activities from the Scot- 
tish Education Department. 

Between 1950 and 1955, the Joint 
Committee produced 40 films, of which 
31 were silent and nine sound, 20 in 
black-and-white and 20 in color. In 
addition to these Scottish productions, 
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the Central Film Library acquires films 
from governmental and commercial 
sources both at home and overseas. The 
total number of films available to 
schools through the distribution center 
was 2750 in 1950 and 3500 in 1955. 
Silent black-and-white films dominate, 
but the proportion of sound and color 
films is rapidly increasing. To cover the 
demand, some 20 prints may be needed 
of a silent film and some five prints 
of a sound film. 


The films are rented directly from the 
distribution center to schools; prints are 
rarely sold. Regional or local film 
libraries hardly exist. Local school au- 
thorities do not keep a special film 
budget, but they receive from the Educa- 
tion Department a 60 percent subsidy 
for film expenditure. 

The schools possess approximately 
1400 16mm film projectors, of which 
some 1000 are silent and some 400 
sound. The Education Department 
grants a 60 percent subsidy to school 
purchase of projectors via the local 
school authorities. Methods of projec- 
tion vary: films are shown both in the 
classrooms and special film rooms, both 
in darkened rooms and on daylight rear 
projection screens. No film supervisors 
or coordinators exist. 

Most future teachers are given audio- 
visual training by colleges in various 
ways. For inservice teachers, courses 
are arranged by local school authorities 
and local branches of the Scottish Edu- 
cational Film Association. Film expres- 
sion and art are not included as a special 
subject in the study programs. 


United States of America 


There is no country-wide pyramid of 
administration and conditions vary from 
state to state. In some states a pyramid 


has developed from the regional level 
downwards, in other states from the 
local level upwards. 


The production and central distribu- 
tion of educational films for schools is 
undertaken by private enterprise, and 
many producers are active in this field. 
They do not receive any subsidy from 
the authorities. University production 
of specialized films is increasing but as 
yet is insignificant compared with private 
production. 


It has been calculated that between 
1950 and 1955 some 2000 new educa- 
tional film titles have been released each 
year. Approximately 10,000 film titles 
are available to schools, practically all 
sound films. The use of color films is 
rapidly increasing. Six hundred prints 
of a film for elementary schools is a 
fairly normal figure. 


The producers sell prints of films to 
regional and local film libraries. In five 
states, regional film libraries are oper- 
ated as a service of the State Department 
of Education. A number of state uni- 
versity film libraries are financed through 
rental of films on a revolving fund basis. 
Other regional or local film libraries are 
operated by local school authorities, 
sometimes with state government sub- 
sidies. In the latter case, the films are 
available free of charge to schools in 
the area. 


It is impossible to calculate, even ap- 
proximately, the number of 16mm film 
projectors at the disposal of the schools, 
but it is known that about 95 percent 
of the secondary schools and 80 percent 
of the elementary schools own one or 
more sound projectors. Silent projectors 
are hardly used. Most projectors are 
purchased out of school funds which are 
on state government budgets. In many 
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instances, however, the school parent- 
teacher organizations raise funds to buy 
audiovisual equipment. In principle, the 
screening of films takes place in the 
classrooms, but is restricted to special 
film rooms in some schools. Projection 
in darkened rooms is definitely preferred 
to daylight projection. 

In most states, universities, larger 
cities, et cetera, there is a special post 
as director of audiovisual education. An 
increasing number of schools has some- 
one who serves as audiovisual coordina- 
tor for the school. He is normally a 
teacher and is seldom remunerated for 
this work. There is, however, a definite 
trend in the direction of giving the co- 
ordinator some free time. 
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In only two states (California and 
Pennsylvania) is training in the use of 
audiovisual aids compulsory for teacher 
certification. Most teacher training in- 
stitutions, however, offer basic audio- 
visual courses, and today some 30 per- 
cent of teachers entering their profession 
may be expected to have received such 
training. This percentage is increasing. 
The courses often include simple pro- 
duction techniques but very seldom film 
expression and art. A large number of 
summer courses are open to inservice 
teachers, and at least 50 institutions for 
higher education offer training suffi- 
ciently complete to qualify participants 
for posts as directors of audiovisual 
education. 
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THE EUROPEAN APPROACH TO AUDIOVISUAL 
RESEARCH — A CURRENT REPORT 


Nosovy SHOULD BE MISLED into 
thinking that what I propose here is 
some sort of comprehensive European 
survey. I am not really qualified to 
make such a survey, but I will add that 
there cannot be many people who are; 
and even if there were some ideal per- 
son who had it all at his finger-tips he 
would find it difficult, in a relatively 
confined space, to provide more than 
a series of jottings which in toto might 
not perhaps be very informative. 

The first thing to note is that there 
is no coordinated body of European re- 
search. Europe is one continent but 
still many different countries—different 
politically, economically, socially, lin- 
guistically, educationally, and in tradi- 
tion—despite all the relationships be- 
tween them; each country accordingly 
tends to make its own autonomous ap- 
proach to the use of audiovisual aids 
and to the research problems that arise. 
Some have done a good deal in the field 
of research, some relatively little, but, 
in addition, coordination within coun- 


Arthur Vesselo * 


tries is often rudimentary and nowhere 
anything like complete. To attempt in 
any thorough fashion to pick up all the _ 
ends and knit them together into a co- 
herent pattern would itself involve an 
elaborate work of research and exposi- 
tion. It is not material for a short article. 

On the other hand, there are certain 
important points which can be picked 
out and certain generalizations which 
can reasonably be made. For all the 
differences between European countries, 
there is also, it seems to me, a basic 
difference between the United States and 
Europe as a whole in the fundamental 
approach to the use of audiovisual aids, 
particularly in the educational sphere. 
While America cannot claim any chron- 
ological priority in the use, for example, 
of the film for documentary and educa- 
tional purposes (some of the first big 


* The author is associated with the Films 
Division of the Central Office of Information 
in London. He attended the 1959 Lake Oko- 
boji Leadership Conference in lowa where re- 
search was the major topic. Mr. Vesselo rep- 
resented European audiovisual groups. 
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advances were certainly European), 
nevertheless the concentrated drive that 
has taken place in America over the last 
decade and a half towards the intensive 
application of every conceivable audio- 
visual means in every field of instruction 
has no real parallel on the European 
scene. 


Europe vs. U. S. Approach 


I do not mean that there has not 
been plenty of development in Europe 
also, but the attitude is subtly but with- 
out question different. In America there 
has been created a body of clearly-de- 
fined specialists (audiovisual directors) 
operating throughout the educational 
system; and the intensity of the drive, 
as far as mechanical devices are con- 
cerned, has been such that it is begin- 
ning to take on some of the characteris- 
tics of a thoroughgoing technological 
revolution in the realm of education. 
Things have not been going in quite the 
same way in the European countries. 


In Europe new things are not, as a 
rule, seized on quite so avidly, as it were, 
for their own sake. Americans have 
acquired, in the course of their history, 
a tremendous and restless appetite for 
change and for technological develop- 
ment at the greatest possible rate and on 
the vastest scale. European methods 
tend on the whole to be more gradual, 
with a more elaborate adjustment be- 
tween the old and the new—partly on 
general grounds (including a dash of 
scepticism) partly for economic reasons. 
The latter may often limit expenditure 
quite severely, and make careful control 
essential. The result, in the sphere of 
audiovisual instruction, has been a pro- 
cess of stage-by-stage assimilation, with 
considerable variations between country 
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and country—also with much less em- 
phasis on the creation of large numbers 
of full-time AV directors and staffs, and 
with a slower development of the more 
costly mechanical devices. 


It might be thought that the more 
gradual, more cautious approach would 
automatically involve more detailed pre- 
liminary research. This does not, how- 
ever, necessarily follow. Research re- 
quires facilities, and facilities are, to a 
large extent, a matter of the opportuni- 
ties afforded by already existing prac- 
tice. Research projects also have a way 
of being expensive. And they have a 
way, in the short term at least, of being 
at many points somewhat inconclusive. 
Money is more likely to be found for 
them among the converted or semi-con- 
verted than among those who “don’t 
know” (and probably don’t greatly 
care). But where money is short, the 
converted or semi-converted may prefer 
to spend it, if they spend it at all, on the 
development of practice (taking what 
may be described as a calculated risk), 
rather than on prolonged theoretical 
investigations. 


If this is all true, then one would 
naturally expect to find more researches 
being carried out in Anmierica than in 
Europe. This is certainly so in general. 
On the other hand, as I have already 
made clear, there are great differences 
between countries. One country which 
has paid great attention to research 
into the use of AV methods in educa- 
tion is France. I know of no country 
where a more centralized and continuous 
effort has been made in this direction 
since the war. The body responsible for 
the planning and conduct of the re- 
searches is the Audiovisual Center of 
the Superior Normal School at St. Cloud 
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under Robert Lefranc. The Center was 
set up by the Ministry of Education in 
1947, specifically to concern itself with 
(a) scientific research, (b) experimental 
production, and (c) dissemination of 
new AV methods (primarily within the 
teaching profession). Its basic purpose is 
the highly practical one of adapting suit- 
able AV techniques to the particular 
needs of the French educational system. 


Research Aims in France 


Starting clearly from the premise that 
AV techniques in education are a good 
thing (research almost always starts 
from a fairly firm general conviction of 
this kind), the Center uses a variety 
of experimental methods to draw dis- 
tinctions between techniques in particu- 
lar contexts, to improve the effectiveness 
of the techniques in application, and to 
create “teaching norms.” The aims of 
the research, that is to say, are discrimi- 
native, ameliorative, and system-finding. 
These aims are set out by the Center 
more specifically as follows: 

1. To study the [ideal] pedagogical 
value of existing AV media 

2. To study the value of the different 
AV media in relation to the present cir- 
cumstances of teaching in France 

3. To pursue a progressive improve- 
ment in the effectiveness of each AV 
medium 

4. To work out a “doctrine of utiliza- 
tion” of audiovisual aids 

5. To extend the use of audiovisual 
aids into educational fields as yet in- 
sufficiently exploited (for example, mod- 
ern languages, mathematics, civics) 

6. To perfect methods of inexpensive 
production. 

This charting of tasks is interesting to 
note, even though there is unfortunately 
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no room here to examine their signifi- 
cance in detail or to discuss the methods 
which are being used (pilot classes and 
control groups, questionnaires in various 
contexts, psychological and perception 
tests, etc.). Attention may be drawn 
however to (2) and (6) as indicative 
of a general trend. The Center is par- 
ticularly concerned in relating its investi- 
gations to actual facilities and actual 
practice in France, and to think in terms 
of the most economic means available 
for given ends and given circumstances. 
It may be noted also that its operations 
in the realm of experimental production 
do, of course, tie up closely with its re- 
search possibilities. 


Mention should be made, before leav- 
ing France, of the Schools Television 
Service (under Henri Dieuzeide), also 
run by the Ministry of Education 
through the National Pedagogic Insti- 
tute, with which the Center has a close 
working relationship. The Service was 
the first of its kind in Europe (1951), 
though no longer now the largest, and 
therefore deserves a special word. It 
issues four half-hour transmissions a 
week, and about 4000 schools at present 
make regular use of them. 


More recently both Great Britain and 
Italy have come into the field, each in 
its own rather different way (in Britain, 
for example, where a great deal has 
been done in the last two years or so, 
the service is not state-controlled, and 
is divided between the BBC and com- 
mercial television). An operation of 
this sort and on this sort of scale ob- 
viously offers most valuable material for 
trained investigators to check, examine, 
and test. And flourishing practice, as 
already observed, naturally stimulates 
speculation and research. In France, the 
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Center at St. Cloud takes full advantage 
of the existence of the Television Service 
in its research investigations and, further, 
the Institute is at present engaged in 
some closed-circuit experimentation at 
the Lycée in Sévres in connection with 
the planning of programs more di- 
rectly aimed at helping out the teacher 
shortage. 


Much of European AV research ac- 
tivity takes the form of individual and 
temporary projects, undertaken under 
the auspices of particular educational in- 
stitutions and authorities, or other types 
of organization, with particular pur- 
poses in view. The scope and status of 
such researches vary widely. On the 
university level psychological studies are 
often the clue. Sociological interests are 
also an important factor, and here we 
may very often find ourselves getting 
away from experimental research to 
“armchair” theory (I use the term in no 
disparaging sense; nothing is possible 
without a basis of well-argued and stim- 
ulating theory) and from organizations 
to individual theorists. The International 
Institute of Filmology (of French ex- 
traction) and its offshoots have given 
rise to studies ranging through psychol- 
ogy, sociology, even to philosophy— 
dealing most usually with the effects of 
the cinema upon the public, from one 
aspect or another, and having strictly 
educational implications only here and 
there. 


Experimentation in Italy 

A pioneer from the side of psycho- 
logical analysis and experimentation is 
Italy. A recent Unesco bibliography 
lists contributions by M. Ponzo on “psy- 


chological observations made during 
cinematograph presentations” and “the 
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didactic value of the film” as early as 
1911 and 1914 respectively. A good 
deal of valuable work in psychological 
and associated fields has been and is 
being done on the university level in 
both Rome and Milan. In Belgium there 
is some interesting current work on re- 
actions to films by primitive populations: 
the sponsoring bodies are the Psychology 
Department of the Free University of 
Brussels and the National Trust for 
Scientific Research. 


I could go on to make out extensive 
lists of learned and other investigations 
completed or in progress. If I do not 
do so, it is because I feel that this is not 
the place. Lists longer and fuller than 
any I could possibly supply here are 
prepared from time to time under vari- 
ous subject heads by the Clearing House 
of the Department of Mass Communi- 
cation at UNesco, which thus acts to 
some extent as a coordinating body. I 
say “to some extent” advisedly, since 
even the coordinating work of pure and 
simple bibliographic compilation (with 
names, dates, and basic references but 
without description or appraisal) is 
heavy, and it is evident that UNEsco 
(which also organizes occasional semi- 
nars and publishes certain descriptive 
reports) has not found the facilities to 
do it as thoroughly and continuously, 
and as analytically, as it ultimately re- 
quires. 


This passing comment is indeed very 
relevant to the topic under discussion. 
It is an obvious first thought that the 
lack of coordination to which I referred 
at the beginning of this article could be 
remedied through the instrumentality of 
one or other of the international agencies 
which have existed since the war. But 
the problem is not so simple of solution. 
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Not only UNEsco but, for example, the 
European Productivity Agency, and 
lately the main body of OEEC (Organi- 
zation of European Economic Coopera- 
tion) itself, have shown interest in as- 
pects of AV research, but this interest 
may well be limited to a more or less 
particularized aspect at a particular time 
(and may perhaps tend to be schizo- 
phrenically split between practice and 
theory), or, alternatively, it may (as I 
feel with UNESCO) be too wide-ranging 
to be really effective within the limits 
of the manpower available. The fact is 
that a proper, solid, scholarly assembly 
of material from all quarters, kept con- 
tinuously up to date in a useful form and 
disseminated where required, demands 
more organization (and cash?) than is at 
present available, and certainly requires 
a higher degree of working collaboration 
between center (whatever center) and 
periphery than seems possible under 
present conditions. 


As far as this article is concerned, I 
can pretend to do no more than throw 
out a few ideas and pick out a few 
interesting highlights. I am making no 
attempt to apportion accurately accord- 
ing to importance, or even to include 
everything which is most important. I 
am trying merely to give a selective im- 
pression. And here I would like to di- 
verge momentarily from research proper 
in order briefly to draw attention to an 
unusual venture which I think not alto- 
gether irrelevant. 


In Ulm, South Germany, there is 
a college of design (Hochschule fiir 
Gestaltung), founded as a memorial to 
two anti-Nazi students, which includes 
a communication or information division 
—something quite novel in conception. 
As well as copy-writing, poster design, 
typography and so on, the division gives 
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instruction in, for example, scripts for 
film and radio. It goes as deeply as it 
can into communication and information 
theory—in specifically scientific terms— 
with the well defined practical aim of 
getting away from “aesthetic” imponder- 
ables. This is not precisely a matter of 
research as such but it involves a way 
of thinking and working, for whatever 
it may be worth, which has lively poten- 
tialities for research developments in the 
future. And equally, if it is not directly 
related to the question of teaching by 
audiovisual means, it has a powerful in- 
direct relationship. There is something 
here to watch. 


British Efforts Individualized 

I have said almost nothing so far 
about my own country, Britain. In 
Britain, research activities of several 
kinds have been taking place across a 
number of years, but in the main they 
have been of the nature of individual 
and not specifically related projects— 
some short-term, some longer term—but 
not on a continuing basis. There is as 
yet no effective center for the collection 
and collation of research information, 
though published reports appear in vari- 
ous places. Attempts have been made 
to start up such a center (I personally 
took part in one such attempt just after 
the war), but funds and a proper home 
for it have proved difficult to procure. 
There is a small Communications Re- 
search Centre at University College 
London; which may have future possi- 
bilities in our field, though currently its 
chief pre-occupation is with language. 
Meanwhile the first British Research’ 
Chair in Communication, endowed by 
Granada Television, at the University 
College of North Staffordshire, has just 
been filled—by a physicist. 
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Some useful experimental and ana- 
lytic work has been done. Some of it 
has naturally emanated from the uni- 
versities. An example of a few years 
back worth recalling—a small model of 
its kind for the care and detail put into 
its preparation and analysis—was the 
experiment carried out in 1944 in four 
Exeter senior schools (by G. Patrick 
Meredith, then at the University College 
of Exeter, now Professor of Psychology 
at Leeds) to help towards defining the 
respective provinces of visual and oral 
teaching methods. It was published in 
1950. 


At the present time the National Com- 
mittee for Visual Aids in Education, the 
central promotion and procurement body 
for schools since 1948, is conducting 
some interesting experiments with AV 
aids for use in teaching backward chil- 
dren. Big industrial concerns have oc- 
casionally initiated inquiries of a limited 
kind. My own organization, the Cen- 
tral Office of Information, has a Social 
Survey division which has been respon- 
sible for one or two particular investiga- 
tions and has published certain material 
on children and the cinema. I myself 
also handled recently (1958) for our 
Films Division an investigation into the 
comparative effectiveness of two kinds 
of short films for recruitment of school- 
boys into the Armed Services. The ap- 
proach was unpretentious (using the 
method of external observations in the 
context of the films’ normal use—with 
questionnaires for the observers only) 
but results seemed genuinely helpful 
from the point of view of future pro- 
duction. 


In 1954 Mary Field undertook, for 
the Carnegie United Kingdom Trust, an 
inquiry into children’s reactions to cine- 
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ma films, using the interesting technique 
of external observation by means of 
infra-red photography (backed up by 
other data). Researches into effects on 
children naturally bulk large in all con- 
nections and not only where AV aids in 
education are under consideration. 
Symptomatic is the very full report on 
Television and the Child, published in 
1959 in book form, resulting from an 
investigation suggested in 1954 to the 
Nuffield Foundation by the Audience 
Research Department of the BBC and 
conducted by Hilde Himmelweit, Reader 
in Social Psychology at the London 
School of Economics, and some col- 
leagues. (Note also that a new British 
Film Institute report, Film and Tele- 
vision in Education for Teaching— 
largely concerned, from the teacher’s 
angle, with film and television as mass 
entertainment media—presses the need, 
among other things, for further audience 
research.) 


Broadcasting, in particular television, 
is perhaps the one field in which there 
is anything like a continuous research 
operation, though largely angled in other 
directions than education as such. Of 
course there is a good deal of energy 
expended both by the BBC and its com- 
mercial television competitors (the latter 
with the aid of independent business 
research organizations) on the measure- 
ment of audience numbers. 


Of considerably more significance for 
our present purpose is the type of care- 
fully controlled before-and-after attitude- 
sampling operation conducted by the 
BBC Audience Research Department 
into television programs or series of pro- 
grams on, for example, mental health. 
A body of valuable experience is thus 
built up. The schools services also have 
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their system of regular feedback from 
the classroom, though as it stands at 
present this is not quite “research” in 
the full sense. In the broadcasting sphere 
in general one sees the possibility of a 
genuine interchange of research views 
and data in the long term between Euro- 
pean countries through the medium, for 
example, of the regular meetings which 
take place between members of the 
European Broadcasting Union. 


| offer my apologies to all those whose 
names or work I have omitted to men- 
tion. | want to finish this all too sketchy 
account by the briefest of words about 
the future of research. It is obviously 
essential that some means shall be found 
of establishing fuller and more sys- 
tematic contacts between research work- 
ers not only all over Europe but all over 
the world. Only so will it be possible 
to establish a satisfactory framework 
for future development. At the same 
time some attention must be paid to 
basic priorities for development. It 
must never, of course, be forgotten that 
AV research is not a thing in itself but 
must always be related to the main aims 
of education as a whole. 


Character of Research Questioned 


Starting from here, I would like to 
record my own strongly held view that 
one of the first things to be done on any 
scale is to investigate thoroughly the 
character of research itself—to classify 
its methods and techniques, to determine 
which of them are most appropriate to 
what types of problem, to examine into 
the degree of scientific accuracy possible 
in given contexts, and to specify safe- 
guards. (To make one simple illustra- 
tive point here, the questionnaire, most 
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popular of research instruments, is in 
fact a very rough-and-ready — perhaps 
too ready — device, open to error at 
many points, and requiring more cau- 
tious handling than it usually gets.) 
Under present conditions too much 
quasi-research takes place, too few “re- 
sults” are authentic. Even where meth- 
ods are impeccable, summary conclu- 
sions drawn (by the researchers them- 
selves) may be wildly misleading. In- 
terpretation of results has, itself, major 
difficulties which have to be clearly 
recognized. 


One of the biggest of all pitfalls is to 
strain after “scientific” certainty where 
it is simply not obtainable. There is 
nothing whatever wrong with an investi- 
gation which aims merely at the greater 
clarification of a working hypothesis. 
Indeed “merely” is too derogatory a 
word. What thought-advances are ever 
possible, without proceeding, by all 
available means of controlled observa- 
tion and experiment, from one working 
hypothesis to another? For that matter, 
what thought-advances are possible 
without a background of pure “arm- 
chair” speculation, providing ideas to 
be tested? 


In our field there is much that is, and 
will long remain by its very nature, 
highly indeterminate. This need in no 
way limit our explorations. But let the 
admission of it guard us against the 
besetting sin of claiming scientific cer- 
tainty for findings that have not in 
reality satisfied the canons of rigorous 
scientific proof. And, in passing, when 
we hear somebody affirm that “it has 
been conclusively demonstrated that 
...” let us reach, if not for our pistols, 
at least for our hats. 
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A REPORT ON SIX RESEARCH STUDIES 


Dr. Oxhandler who also writes on page 
109 reports here on a session conducted 
during the 1960 Convention of the De- 
partment of Audio-Visual Instruction 
at Cincinnati in March. C. Walter 
Stone, director of the Educational Media 
Branch, U.S. Office of Education, gave 
a brief introductory address followed by 
the progress reports summarized here. 
Kenneth D. Norberg, departmental edi- 
tor for AUDIOVISUAL COMMUNICATION 
REVIEW, was chairman of the session. 


, aren THE DEVELOPMENT and im- 
plementation of Title VII (National 
Defense Education Act), Dr. Stone ex- 
plained briefly that Part A calls for 
research in the newer educational media 
while Part B deals with the dissemina- 
tion of these research findings and in- 
formation. Included under the area of 
research are studies on the use and/or 
adaptation of the newer educational 
media for instruction, the training of 
teachers, and a newer approach to the 
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presentation of subject matter. Dissemi- 
nation contracts include surveys, studies, 
and publications which will tell the story 
of instructional materials research as 
well as developing leadership in audio- 
visual instruction. 

Approximately one-half of the proj- 
ects funded by Title VII relate closely 
to television. The second largest cate- 
gory deals with the development of new 
materials and newer systems of instruc- 
tion. The third major group, now 
rapidly increasing, is concentrated in the 
area of research on teaching machines. 
As an aside Dr. Stone indicated that 
many industrial concerns are inquiring 
about teaching machines, companies 
which themselves have developed the 
“hardware” for teaching machines but 
are in need of educational programming. 

Dr. Stone foresees these major devel- 
opments for the future: 

1. A national training laboratory for 
leaders in the instructional materials 
field. 

2. A nationwide demonstration and 
display center. 
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3. A national research information 
agency to collate, organize, abstract and 
distribute research findings. 

4. Coordination of both national and 
regional efforts to develop, experiment 
with, and exchange instructional mate- 
rials. 

5. An increase in the improvement 
of instruction as it relates to teaching, 
research and administration. 

6. A national statistical inventory. 


7. A greater impact of the newer 
educational media on national program- 
ming and planning. 

8. The development of regional re- 
search centers. 


Dr. Stone concluded with the remark 
that American education faces great 
change, and that in the center of this 
change will be the audiovisual educators 
who will be called on to provide “more 
and better learning for more people in 
less time.” 


Progress reports limited to six min- 
utes each by the chairman revealed the 
following developments: 


John R. Brugger reported on A Sur- 
vey of Television Equipment and Facili- 
ties Used for Purposes of Instruction by 
Public Schools, Colleges and Universi- 
ties (Washington County Public Schools, 
Maryland). Mr. Brugger listed 13 
achievements TV can bring to instruc- 
tion, among them, the efficient use of a 
specialist, bringing outside events into 
the classroom, directing student atten- 
tion through camera application, and so 
on. Three basic types of installations 
were discovered to be in use, namely, 
standard broadcast stations, closed-cir- 
cuit systems, and demonstration ampli- 
fiers. The types of programming ranged 
from training TV personnel to direct 
instruction in the classroom and in- 
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cluded general information programs 
broadcast as a public service. 


As Mr. Brugger sees the problem, 
educational television will need: (1) an 
aesthetic and artistic approach to main- 
tain equality with commercial television; 
(2) special training and assistance for 
the television teacher; (3) trained, ex- 
perienced technicians, including cam- 
eramen, communications _ specialists, 
graphics men, and others, and (4) an 
educated viewer (the student as a critic). 

Basic deterrents to good program- 
ming are lack of funds for staff, lack of 
release time for the television teacher, 
and the expenditures of time and mate- 
rials needed for good production. 


The trend, according to Mr. Brugger’s 
survey, indicates that television will help 
improve education both qualitatively 
and quantitatively. 


Carolyn Guss reported on A Study 
to Determine a Feasible Method of 
Establishing Bibliographic Control of 
Educational Audio-Visual Materials for 
the Purpose of Informing Teachers Con- 
cerning Available Materials and Their 
Educational Utility (Indiana Univer- 
sity). To accomplish the purposes of 
the study, the following steps were 
taken: (1) an inventory of all tools and 
information on newer media was made, 
(2) sources of information were listed 
and checked, (3) persons interested in 
improving and creating new means were 
contacted, and (4) recommendations 
were solicited and made. 


Five compilations of lists were con- 
sulted and a working bibliography of 
117 items was circulated among 149 
qualified people along with a question- 
naire asking for additions and correc- 
tions. The list is now being revised 
according to the 100 replies received. 








152 


The following set of criteria for the 
audiovisual sources bibliography has 
been established: 

1. No items dated prior to 
shall be included. 


2. Items will stress the newer media. 


1950 


3. Materials for elementary and sec- 
ondary teachers will be stressed. 

4. No film or AV library catalogs 
will be listed. 


5. No producer handouts. 


6. All publications where the supply 
is definitely limited, obsolete, or out-of- 
print will be eliminated. 

The investigators have already anno- 
tated over 100 sources of information 
to provide background on the newer 
media for instructional purposes. Miss 
Guss reported that the completion of 
Phase One of the study should be 
accomplished by March 30, 1960. 


Walter T. Martin of the Institute of 
Community Studies, University of Ore- 
gon, presented a progress report on The 
Study of Resistance to Television for 
Educational Purposes. The study was 
set up in a three-year pattern utilizing 
an interdisciplinary social science ap- 
proach. The first year is designed to 
investigate the habits of TV viewers, 
ascertaining what types look at educa- 
tional television, and so on. The inter- 
view method will be used with 500 male 
and 500 female adults as subjects. The 
second year will include sub-studies of 
controlled manipulation of both produc- 
tion and viewing of ETV. During the 
third year, the original 100 viewers will 
be interviewed to determine changes in 
their patterns of behavior. 


The study is now localized in Eugene, 
Oregon and the 1000 persons have been 
randomly selected from a group of 4500 
families who receive four commercial 
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and one ETV stations via co-axial cable. 
This pattern insures equal pictorial qual- 
ity on all five channels thus eliminating 
the variables of picture quality, and 
other factors. At present, less than half 
of the 1000 subjects have been inter- 
viewed and no compilations are as yet 
available. However, the researchers re- 
port asking significant questions and re- 
ceiving replies of both theoretical and 
practical importance. 


Herbert Schueler reported on the 
Hunter College experiment in The Use 
of Television for Improving Teacher 
Training and for Improving Measures 
of Student Teaching Performance. The 
assumption is that closed-circuit tele- 
vision and videotape will improve 
teacher training. 

Phase One, the student teacher phase, 
allows the student teacher to watch 
herself in action through kinescope re- 
cordings. The experiment including 24 
volunteer student teachers has one con- 
trol group with a supervisor, a group of 
student teachers who are supervised 
solely by kinescope recordings, and a 
third group who have both supervision 
and kinescopes. A set of testing instru- 
ments are being developed to (1) deter- 
mine what is good teaching and (2) 
measure this good teaching as practiced 
by student teachers. It is hoped that 
repeated showings of teacher-pupil re- 
action will aid in developing these in- 
struments. Four elementary classrooms 
are now equipped for transmission. The 
TV intrusion is minimized by remote 
control procedures. Cameras are set at 
bulletin board level on the sides of 
classrooms and on shelves. Two cardiod 
microphones, angularly placed, are used 
for classroom pick-up. Extra banks of 
fluorescent lights are provided for added 
illumination. 
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The second phase is an attempt to 
determine whether elementary methods 
courses can be improved by the use of 
edited television recordings of child ob- 
servations. The viewing rooms are so set 
up that the instructor can control the 
observation. Dr. Schueler stated that 
the study planned to develop over 1000 
kinescopes of good teaching practices. 
He also said that the human factors are 
now positive and that student teachers 
like the process very much. 


An Experimental Study of Patterns 
for Improving the Preparation of Pre- 
Service Teachers in the Use of Audio- 
Visual Materials and of the Effects of 
Optimal Use of Audio-Visual Materials 
upon Pupil Learning and Teacher Use 
(Pennsylvania State University) was re- 
ported on by G. M. Torkelson. The 
study is divided into four main patterns 
with students randomly divided into: 
(1) a separate course in AV; (2) an 
integrated course with both elementary 
and secondary methods; (3) a course 
given as part of student teaching both 
saturated with equipment and materi- 
als and non-saturated; and, (4) self- 
teaching. 

Testing materials including perform- 
ance and attitude information, five kine- 
scopes, and an entire self-taught course 
have been developed during the first 
semester of operation. At present, those 
students who have experienced the self- 
teaching pattern have expressed a dis- 
tinct liking for this procedure. Their 
test scores indicate that there is a sig- 
nificantly higher level of performance on 
certain projection equipment than that 
resulting from other patterns. There 
were no significant differences among 
patterns as to the effect of informational 
or theoretical applications of audiovis- 
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ual instruction. It was pointed out, 
however, that the first semester’s work 
was a “dry-run” situation with revisions 
suggested for succeeding semesters. 


The final phase of the study will be a 
follow-up of all those pre-service teach- 
ers who eventually take positions in 
Pennsylvania. Measures will be devel- 
oped to assess the utilization of audio- 
visual materials made by teachers who 
have been exposed to the various pat- 
terns and of the obstacles to the use of 
these materials in the public schools. 


Edgar L. Lowell presented a sum- 
mary of An Investigation of Audio-Vis- 
ual Techniques to Improve the Parents’ 
Role in the Education of Deaf Children 
(John Tracy Clinic, Los Angeles). As 
stated by Dr. Lowell, the basic objec- 
tives of this research is to evaluate 
audiovisual methods for influencing pa- 
rental attitudes toward education. The 
population is being drawn from pre- 
school deaf children and their parents. 


The major variables are: (1) paren- 
tal information and readiness to accept 
change, (2) parental feelings as a com- 
ponent of attitude changes, (3) the 
relative prestige of a “Hollywood” celeb- 
rity as opposed to an “expert” in the 
field, and (5) the use of films, tape 
recordings and printed materials. 


At present, tests are being constructed 
to measure attitude changes. The popu- 
lation is cooperating excellently as are 
the “Hollywood” personalities and the 
film and TV studios. 


Oliver L. Lilley of the School of Li- 
brary Science, Columbia University, told 
of the progress he is making on a Fea- 
sibility Study Regarding the Establish- 
ment of a New Educational Media 
Research Information Service. Many 
samples have been checked in this study 








154 


in which Dr. Lilley is investigating the 
adequacy of present bibliographic con- 
trol. Among periodical indexes, the 
Education Index proved the best while 
Psychological Abstracts rated high 
among abstracting journals. In most 
cases, the bibliographies were quite in- 
complete. With the search of 25 tool 
controls, 17 percent of the newer edu- 
cational media were not even listed. 
Many duplications of listings were also 
discovered. 

Results of a questionnaire indicated 
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that most people looking for informa- 
tion rely most highly on personal com- 
munication and only slightly on libraries. 
Most standard resource tools are either 
not known or known but not used. 
The study indicates that short infor- 
mational abstracts and critical evalua- 
tion summaries are desired most by 
researchers. Multiple services and both 
broad and narrow approaches to the 
search for materials are probably 
needed. Dr. Lilley expected the final 
report to be printed by May 15, 1960. 


SOME SUGGESTIONS REGARDING THE 
PREPARATION OF RESEARCH PROPOSALS 


L. SOMEWHAT less than two years of 
operation, the educational media re- 
search program authorized in Part A, 
Title VII, National Defense Education 
Act (P.L. 85-864) has provided finan- 
cial support to 81 new research projects. 
These projects were selected from 421 
proposals formally submitted prior to 
the September 1959 deadline and when 
each of the approved projects is com- 
pleted, they will have received Federal 
grant funds totalling approximately 
$6,120,000. 

Although there are no clearly identi- 
fiable “secrets of success” for preparing 


Warren F. Seibert 
and Kalmer E. Stordahl * 


proposals there are some important 
policies to be considered in applying 
for a research grant. One of these, as 
already implied, is that to date only the 
grant research provisions of Title VII 
have been activated; contractual author- 
ity for the support of research has not 
been implemented. Grants are awarded 
only to public or non-profit private 
agencies or organizations (colleges, uni- 
versities, public school systems, state 





* Kalmer Stordahl is research coordina- 
tor, Educational Media Branch, U.S. Office of 
Education. See another article by Dr. Seibert 
on page 115. 
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departments of education, or other non- 
profit agencies). Although the Act also 
authorizes the Office of Education to 
make grants to individuals, at present 
it seems likely that grants will continue 
to be made only to the institution or 
agency with which the individual appli- 
cant is affiliated. 

With respect to equipment the Title 
VII policy seems essentially to be 
“middle-of-the-road.” It does not per- 
mit Federal support for permanent in- 
stallations, structural modifications, and 
so on, but it does provide limited sup- 
port for new and “portable” equipment 
(for example, a film projector or TV 
receiver) which is otherwise unavailable 
but which is necessary for the satisfac- 
tory completion of an approved project. 
Support may be provided at the rate of 
20 percent of undepreciated equipment 
value for each year of use in the project. 

Support for indirect costs of a project 
can be included in a grant so long as 
this does not exceed 15 percent of the 
Federal funds granted for support of 
personnel, services, supplies, and other 
direct costs. For example, if an ap- 
proved project called for $50,000 of 
Federal funds to support direct costs, 
an allowance of $7500 could be made 
to cover at least a portion of the indi- 
rect costs (for example, for space, heat, 
power, and general administration). 

A small grants program has recently 
been established as a part of the Title 
VII research activity. The principal 
purposes of this program are to encour- 
age small-scale pilot or exploratory 
studies and to encourage more active 
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participation in media research by 
young or less experienced researchers, 
including graduate students. In general, 
the policies and procedures of the new 
small grants program will be the same 
as those of the larger program; however, 
small grants will be limited to projects 
having a duration of one year or less 
and requiring Federal support of no 
more than $2500. 

In preparing a research proposal, the 
applicant should observe insofar as pos- 
sible the instructions prepared by the 
Office of Education. Copies of these 


_ instructions and of other pertinent ma- 


terials can be obtained by addressing a 
request to the Director, Educational 
Media Branch, U.S. Office of Educa- 
tion, Washington 25, D. C. It is espe- 
cially important to examine carefully 
and bear in mind the five criteria used 
in the evaluation of each proposal. 
These are (1) relevance to the purposes 
of Title VII, (2) educational signifi- 
cance of the problem(s) to be studied, 
(3) adequacy of research procedures, 
(4) qualifications of key personnel and 
adequacy of available facilities, and (5) 
economic efficiency. 

Two proposal submission deadlines 
were established for calendar year 1960; 
the first was February 1; the second is 
August 1. Proposals meeting the first 
deadline were considered by the Title 
VII Advisory Committee at its May 
meeting; decisions concerning proposals 
that meet the August deadline will be 
made when the Committee convenes in 
the fall, probably in late October or 
early November. 








@ FILMS 


Twyford, Loran C. 


“Operations Research on Instructional 
Films.” Journal of the Society of Mo- 
tion Picture and Television Engineers 
68:375-78, June 1959. 


Purpose: To integrate film research find- 
ings into a form that can assist in the 
planning for new film types and new 
equipment that will most adequately 
meet the needs for instructional films. 


Procedure: Factors in the design, produc- 
tion, and utilization of instructional films 
were listed and evaluated subjectively 
for their relative importance. Instruc- 
tional film and television research pro- 
vided the guidance for evaluating the 
relative importance of media factors 
such as sound, picture, motion, and 
attractiveness; production factors which 
include complicated production, costs, 
types of production, and low-cost pro- 
duction; and utilization factors which 
include use of films for essential instruc- 
tion, filmed courses, room darkening, 
film rental cycle, and projector limita- 
tions. 


Results: (1) Sound or verbal material in 
a film appears to be more important 
than the picture or visual presentation 
providing motion. (2) For the most 
part, attractive devices in instructional 
films can be eliminated. Low cost sound 
film recordings can be very effective. 
(3) Providing motion in a film increases 
production costs and complexity. The 
high cost of films can best be justified 
when the films are used for essential 
instruction rather than as aids. (4) 
Using a film is not an easy operation. 
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Obtaining a film involves an excessive 
amount of intercommunication. (5) A 
model instructional film system would 
be ideal. This would involve a simpli- 
fied projector and inexpensive films that 
could be produced locally. Films should 
have greatly expanded use for essential 
instruction. —L. TWYFORD 


Wittich, W. A.; Pella, Milton O.; 
and Wedemeyer, C. A. 


The Wisconsin Physics Film Evaluation 
Project. Madison: University of Wis- 
consin, 1959. 30 p. 


Purpose: To determine the effect of the 
Harvey White physics film course on 
students’ and teachers’ learning of phy- 
sics. To study the opinions of students 
and teachers toward using the series of 
films as part of the regular class in- 
struction and to discdOver whether an 
interest in science was developed. 


Procedure: One hundred sixty-two 30- 
minute film lessons on high school phy- 
sics were used in the study. Eighty- 
three schools participated in the study 
and were divided into an experimental 
and a control group and another con- 
trol group to check on any possible 
“Hawthorne effect.” Lesson outlines 
were prepared for students and teachers; 
a number of tests on knowledge of 
physics were administered to and ob- 
tained from both. 


Results: A test which included text and 
film items showed no significant differ- 
ence between experimental and control 


[156] 








RESEARCH ABSTRACTS 


groups. The experimental group was 
superior in learning film-only items. The 
Ohio test scores were no different. The 
control group retained more after three 
months. The “Hawthorne effect” ap- 
peared to increase the score of the con- 
trol group. The training of teachers 
was directly related to test scores. The 
more credits a teacher possessed the 


@ SOUND FILMSTRIPS 


Dworkin, Solomon, and Holden, Alan 


“An Experimental Evaluation of Sound 
Filmstrips vs. Classroom Lectures.” 
Journal of the Society of Motion Picture 
and Television Engineers 68: 383-85, 
June, 1959. 


Purpose: To find out whether the sound 
filmstrip is a good substitute for the 
classroom teacher. 


Procedure: Four sound filmstrips on the 
subject of atomic bonding were pro- 
duced. They were shown to one group 
while a second group was taught by the 
regular classroom lecture. One hundred 
twenty graduate engineering students 
made up the two matched groups. Each 
of the four filmstrip lectures contained 
about 150 frames and ran for about 50 
minutes. The teacher who gave the 
classroom lecture also wrote and re- 
corded the filmstrip scripts. Sound was 


@ TELEVISION 


Carner, Richard L. 


Teaching Reading Through Closed-Cir- 
cuit Television in the Elementary School. 
Division of Education Communications, 
New York State Education Department, 
Albany, November 1959. 11 p. 
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more he learned. Teachers learned a 
significant amount from using the film 
course. They felt the films were ex- 
cellent but they did not wish to step 
aside for the “film teacher.” Student 
opinions were varied. The experimental 
group had more interest in science but 
did not look forward to seeing the 
films. —L. TWYFORD 


recorded on an inexpensive tape re- 
corder. An examination and question- 
naire were administered after the four 
presentations. 


Results: There was no significant differ- 
ence in learning between the two meth- 
ods of teaching. Matching of groups was 
adequate. The rate of delivery of ma- 
terial was felt to be too fast. Forty 
percent indicated they would like to ask 
questions; 75 percent were willing to 
accept sound filmstrips for instruction. 
However, this result may apply only to 
graduate engineering students. It was 
concluded that sound filmstrips might 
be substituted for more expensive films 
or television and that other methods 
such as reading assignments might be 
substituted for the classroom lecture. 
This study confirmed previous research 
studies which indicate that elaborate 
production methods, including color, 
are not necessarily needed for effective 
learning. —L. TWYFORD 


Purpose: To determine the effectiveness of 
teaching reading at the elementary level 
through closed-circuit television. 

Procedure: Reading was taught over tele- 
vision three times a week for 28 weeks. 
The experimental group was made up 
of 198 pupils in the fourth and sixth 
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grades and the control group of 160 
pupils. Growth in reading was meas- 
ured using two forms of the Iowa Tests 
of Basic Skills. A questionnaire was 
given to study attention span, reaction 
to talkback, degree of interest in the 
lessons, amount of free reading, and 
word attack skills. Teachers used mate- 
rials produced or created for their tele- 
casts as well as commercially prepared 
materials. 


Results: The television group was found 
to do significantly better than the con- 
trol group in spite of the fact that the 
control group appeared to be signifi- 
cantly superior intellectually. Vocabu- 
lary was more improved than compre- 
hension. A 30-minute telecast is prob- 
ably more adequate than the 20-minute 
one. The questionnaire seemed to indi- 
cate that the pupils felt that television 
was a valuable educational medium. 

—tL. TWYFORD 


Murnin, Joseph A. 


The Development and Evaluation of a 


Low-Cost Closed-Circuit Instructional 
Television System: The One-Camera 
Television Chain. Technical Report 


NAVTRADEVCEN 5-11-1. Research 
by The Pennsylvania State University 
for the U. S. Naval Training Devices 
Center, Port Washington, New York, 
December 18, 1958. 90 p. 


Purpose: To develop and evaluate a low- 
cost closed-circuit educational television 
system for application to mass military 
training. The system was to employ a 
single vidicon camera chain and enough 
receivers to instruct 400 trainees. It was 
to be kept sufficiently simple so that 
operation and maintenance could be 
handled by personnel normally assigned 
to a training command. The research 
was directed toward finding a practical 
and effective application of television 
for instructional purposes which would 
(1) have the advantages of increased 
size and extended use of good instruc- 
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tors; (2) provide a greater incentive for 
improved instruction; and (3) include 
the use of training aids. No change was 
contemplated in the content of the 
course to be taught. Plans called for 
some comparison of academic achieve- 
ment. The study was to obtain some 
indication of acceptance by administra- 
tors, instructors, and trainees in terms 
of interest, motivation, criticisms, and 
attitudes toward televised instruction. 


Procedure: The Indoctrination Curriculum 
of the Recruit Training Command, Bain- 
bridge, Maryland was selected for the 
research. Fifty-seven lessons which are 
normally non-visual were chosen. A 
single camera fed a signal to an audio- 
video mixer and the RF signal was sent 
to four classrooms, each having three 
24-inch receivers. A zoom lens and a 
lavalier microphone were used. Four 
750-watt spots and two fluorescents pro- 
vided 200 foot-candles of light. The 
total cost of the installation was $7,872. 
Regular personnel operated the equip- 
ment and put on the lessons. The one 
camera was directed to the instructor, 
visual display table, overhead projector 
screen, or the motion picture screen as 
required by the lesson. A regular pro- 
jector with the shutter removed pro- 
jected a satisfactory picture on a rear- 
projection screen for television trans- 
mission. A number of visual aids were 
adapted to television use. A total of 
11,175 student-hours of televised in- 
struction was presented during one 
week. Multiple-choice test items were 
given to experimental and_ control 
groups selected at random. All persons 
involved in the project made subjective 
evaluations. 


Results: The low-cost, closed-circuit, one- 
man television system was shown to be 
effective. The technique of using a 
single camera to provide transitions 
from scene to scene makes the system 
practical for instructional purposes. 
Four companies of recruits can be 
taught simultaneously and additional 
companies can be added for little extra 
cost. Station-wide coverage appears to 


be feasible and would permit moving 
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the instruction rather than the men. 
Normally assigned personnel with no 
prior experience can be trained to oper- 
ate, maintain and instruct in a brief 
time. Thus, only an instructor and 
cameraman are required to present a 
complete lesson. A standard sound pro- 
jector with the shutter removed provides 
a convenient and inexpensive television 
film projector. The entire Indoctrina- 
tion Curriculum can be adapted to tele- 
vision. Instructors see the advantages 
and prefer televised instruction. No 
differences were found in achievement 
scores. —L. TWYFORD 


The Fund for the Advancement 
of Education 


The National Program in the Use of 
Television in the Public Schools. A Re- 
port on the First Year 1957-58. 477 
Madison Avenue, New York 22, Janu- 
ary 1959. 89 p. 


Purpose: To discover whether television 
can be used to teach very large classes 
at substantial savings in classroom space 
and in teaching positions while at the 
same time improving the quality of 
education. 


@ TRAINING DEVICES 


Tufts College 


Coordinate Index and Abstracts of 
Training Device Literature. Abstracts 
prepared by Tufts College Institute for 


Applied Experimental Psychology. Co- — 


ordinate Index prepared by Documen- 
tation Incorporated, NAVEXOS P-1735. 
U. S. Naval Training Devices Center, 
Human Engineering Department, Port 
Washington, New York, December 4, 
1957. 212 p. 
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Procedure: Superintendents of large city 
school systems were contacted and a 
nation-wide experiment was set up in- 
volving 40,000 students in more than 
200 elementary and secondary schools. 
Thirteen experiments were undertaken. 
Television classes were usually held in 
the school auditorium. The elementary 
classes normally spent half a day in 
small groups and the other half in 
groups of 75 to 150. In junior and 
senior high school classes the television 
instruction was usually limited to a sin- 
gle course. Achievement was measured 
by standardized tests. 


Results: Of the comparisons between con- 
trol and experimental groups, 68 fav- 
ored television and 42 favored control 
classes. Where the differences were sig- 
nificant, 29 were in favor of television 
classes and nine in favor of the control 
groups. Television proved to be valu- 
able for inservice training of teachers. 
Several school systems reported substan- 
tial savings. Dade County estimated a 
savings of $250,000 on cutting the em- 
ployment of teachers by 60. However, 
there were major problems involved, 
among which are inadequate facilities, 
finding good television teachers, training 
teachers in handling large classes, inte- 
gration of the telecast, and scheduling. 

—L. TWYFORD 


Purpose: To abstract all technical reports 
and research studies related to the 
theory, design and use of training de- 
vices and audiovisual aids from 1950 
through 1955. In order to facilitate 
comprehensive searching of the ab- 
stracts, the bibliography was to be pub- 
lished in a form especially designed for 
the use of multiple cross-referencing. 

Procedure: Tufts College developed a 
classification system for relating re- 
search in various fields. Sixteen hundred 
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abstracts were prepared from the re- Results: About 70 percent of the unclas- 


search articles and bulletins. Each ab- 
stract contained about 80 words and in- 
cluded important words related to the 
content. In addition each abstract was 
accompanied by a citation of author, 
title, contract number, report number, 
date, pages, and source. After consid- 
ering several methods for retrieving in- 
formation from the bibliography, a co- 
ordinate index was selected. 


sified literature for the period was ab- 
stracted. The 1600 abstracts and cita- 
tions were published in a book with the 
coordinate index. To find a research 
article dealing with color in posters, for 
example, one compares numbers under 
the word color with those under posters. 
Reports were also listed by sources and 
authors. 

—L. TWYFORD 











